Showing posts with label Monitoring tools. Show all posts
Showing posts with label Monitoring tools. Show all posts

Thursday, October 30, 2014

Configure awstats with apache on solaris


  Introduction : 

awstats

Awstats(Advanced web statistics) is an open source log analyzer. It can analyze a wide variety of logs with different log format of many different web servers. This tool can be configured in both windows and UNIX environments. 

This Log Analyzer works as a CGI or from command line and shows all the possible information that the log contains, in few graphical web pages. It uses a partial information file to be able to process large log files, often and quickly.

Downloading Awstats : 


Awstats can be downloaded from the following URL :

Installing and configuring AWStats on Solaris :


Following are the simple steps to install and configure awstats with Apache logs. 

Step 1 : 


Extract the downloaded awstats (awstats-7.0.tar.gz) using the following command.

gtar -xzf awstats-7.0.tar.gz

The folder awstats-7.0 will be created now.

Step 2 :


Navigate to awstats-7.0/tools and hit the following command.

perl awstats_configure.pl

Following configuration related questions will pop-up.

-> If you want to install the awstats in a customized directory, Then type "y" and hit enter.

 If yes, Then a new awstats configuration file will be created.

-> Enter the name of your application or the profile name. ex : test1

You can give any name that you prefer. This name will be used in the url that is used to access the awstats.

-> Enter the path of the folder, Where the configuration files are to be stored.

This path will contain the awstats.configuration file.

-> After this the screen will exit.

There will a sample config file created in the path chosen during the configuration. We can amend that configuration in sync with our needs.

Step 3 : 


The Main elements to change in the awstats.test1.conf : 

LogFile="/var/log/httpd/mylog.log" 

--> Define the access.log path of the Apache web server.

LogFormat="%host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"               

--> This LogFormat has to be in sync with the Apache logs logging format.

SiteDomain="test1" -- This SiteDomain value will be used in the URL as following.

HostAliases=" 127.0.0.1 localhost <servername>"

--> All other possible domain names, Addresses and ip's that you want to access your site.

DirData="/opt/app/gordon/awstats"

--> When the Awstats updates it statistics, it stores the results of it's analysis in files(Awstats database). All these files will be written to the directory specified in DirData.

AllowToUpdateStatsFromBrowser=1

--> This update will provide a link <update> in the browser, By clicking this link, The Awstats data will be refreshed.



Step 4 : 


Generate the statistics for the first time by using the following command.

perl awstats.pl -config=test1 -update

AWStats will read the configuration file awstats.test1.conf and create or update its database with all summary information issued from analyzed log file.

AWStats statistics database files are saved in directory defined by the DirData parameter in configuration file.

When the create/update is finished, you should see a similar result on your screen :

Update for config "/awstats/awstats.test1.conf"
With data in log file "/pathtothelog/logfile.log"
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 225570
 Found 125 dropped records,
 Found 97 corrupted records,
 Found 0 old records,
 Found 225321 new qualified records.

Dropped records are records discarded because they were not "user HTTP requests" or were requests matching AWStats filters

Corrupted records are records that do not match the log format defined by the "LogFormat" parameter in the AWStats configuration file.

Old records are simply records that were already processed by a previous update session. Although it is not necessary to purge your log file after each update process, it is highly recommended that you do so as often as possible.

New records are records in your log file that were successfully used to build/update the statistics database.

Step 5 : 


Statistics can now be accessed by using the following url from the browser.

Step 6 : 


The statistics updation script can be added in the cronjob for automatic updation.

40 * * * * /Pathtothescript/update_stat >/dev/null 2>&1

Update_stat will contain the following command : 

perl awstats.pl -config=test1 -update


                                                                         (: Happy learning :)

Easy steps to Install and configure JAVA Melody in tomcat - Solaris

This summary is not available. Please click here to view the post.

Configure qslog with Apache (2.2.24)


Description:

The qslog tool, which is part of the support utilities of mod_qos, may be used to gather request statistics from Apache's access log data. This includes data such as the number of denied requests or new VIP session creations per minute but also total requests per second and other data.

These support utilities will be available in the src folder of mod_qos installable.

ex:  /softwares/mod_qos-10.29/tools/src contains qslog,qslog.c,qslog.o

Installation and configuration of qslog:

In case, If you have not configured mod_qos.so module with Apache, then click on steps to install and configure mod_qos with Apache 2.2.24.

Following are the steps to install and configure the qslog with apache: 
  
export LD_LIBRARY_PATH=/usr/lib:/usr/sfw/lib
export LD_LIBRARY_PATH_64=/usr/lib/64:/usr/sfw/lib/64
export CC=gcc
export CFLAGS='-m64 -O3'
export CPP_FLAGS='-m64 -O3'
export LDFLAGS="-L/usr/sfw/lib -R/usr/sfw/lib"

./configure --prefix=/apache_path --with-apr=/apr-1.4.6 --with-apr-util=/apr-util-1.4.1 --with-pcre=/pcre

make

make install

After the completion of make install the qslog script will be created in bin folder of the apache.

Making install in src
test -z "/opt/web/apache_fp/apache/bin" || .././install-sh -c -d "/apache/bin"
  .././install-sh -c qsfilter2 qslog qspng qsrotate qssign qstail qsgrep qsexec qscheck qsgeo qslogger qshead 'apache/bin'

Configuration in the httpd.conf log:

LogFormat "%t %h \"%r\" %>s %b \"%{User-Agent}i\" %T"
TransferLog "|/apache/bin/qslog -f ..IRSB.T -o /apache/logs/stat_log"

Errors:

Incase if the following error is seen in the logs after installing the apache

piped log program 'apache/bin/qslog -f ..IRSB.T -o /apache/logs/stat_log' failed unexpectedly
ld.so.1: qslog: fatal: libapr-1.so.0: open failed: No such file or directory
Killed

Add the APR and APR lib folder path to the LD_LIBRARY_PATH variable.


                                                                         (: Happy learning :)






Thursday, March 27, 2014

Install and configure mod_qos with apache - Limit the number of concurrent requests using apache

Description:

In computer networking, the term quality of service (QoS) describes resource management rather than the quality of a service. Quality of service implements control mechanisms to provide different priority to different users, applications, and data connections. It is used to guarantee a certain level of performance to data resources. The term quality of service is often used in the field of wide area network protocols (e.g. ATM) and telephony (e.g. VoIP), but rarely in conjunction with web applications. mod_qos is a quality of service module for the Apache web server implementing control mechanisms that can provide different levels of priority to different HTTP requests.

Installation and Configuration:

Installation and configuration of mod_qos requires Openssl.

The steps to install PCRE,APR,APR-UTIL and Openssl are included as well.

Apache version : 2.2.24, Mod_qos version: 10.29

export  PATH=/usr/ccs/bin:/usr/sbin:/usr/bin:/usr/sfw/bin:/usr/sfw/sbin:
export LD_LIBRARY_PATH=/usr/lib:/usr/sfw/lib
export LD_LIBRARY_PATH_64=/usr/lib/64:/usr/sfw/lib/64

For 64 bit installation:

export CC=gcc
export CFLAGS=-m64 -O3
export CPP_FLAGS=-m64 -O3

PCRE Installation: 

cd /softwares/pcre
./configure --disable-cpp CFLAGS="-g -O3" CC="gcc -m64" --prefix=/apache/pcre
gmake clean
gmake
gmake install

Openssl Installation:

cd /softwares/openssl-1.0.1f
./config --prefix=/apache/openssl-1.0.1f thread shared solaris64-gcc -m32
gmake clean
gmake
gmake install

APR Installation:

cd /softwares/apr
./configure --prefix=/apache/apr-1.4.6 --with-gnu-ld --enable-threads
gmake clean
gmake
gmake install

APR-UTIL Installation:

cd /softwares/apr-util
./configure --prefix=/apache/apr-util-1.4.1 --with-openssl=/apache/openssl-1.0.1f --with-apr=/apache/apr-1.4.6 --with-crypto --enable-threads
gmake clean
gmake
gmake install 

Apache & Mod_qos Installation:    


tar -xfz httpd-2.2.24.tar.gz 
tar xfz mod_qos-10.29-src.tar.gz
ln -s httpd-2.2.24 httpd
cd httpd
mkdir modules/qos 

cp ../mod_qos-10.29/apache2/* modules/qos

./buildconf - You have to install libtool,m4 and autoconf utilities and provide their path in PATH variable.

cd /httpd

./configure --prefix=/apache --enable-so  --enable-module=all --enable-mods-shared=all --enable-proxy --enable-ssl --with-ssl=/apache/openssl-1.0.1f --enable-vhost --with-mpm=worker --enable-qos=shared --enable-unique-id --with-pcre=/apache/pcre --with-apr=/apache/apr-1.4.6 --with-apr-util=/apache/apr-util-1.4.1 

Incase, If the apache is already installed use the following command from qos directory of src in apache.

./configure --with-apache=/apache_path  --with-mpm=worker --enable-qos=shared --enable-unique-id --with-pcre=/apache/pcre --with-apr=/apache/apr-1.4.6 --with-apr-util=/apache/apr-util-1.4.1 --with-openssl=/apache/openssl

gmake clean

gmake

gmake install
 

Mod_qos Configuration:

After running the make install, The mod_qos.so module will the be automatically configured in the Apache configuration using the following directive.(httpd.conf)

LoadModule qos_module modules/mod_qos.so

Depending on the requirement,The parameters can be edited.

QS_ClientEventLimitCount 5 1
QS_SrvMaxConnPerIP 5
QS_Limit_Clear
QS_ErrorResponseCode 505

                                                                         (: Happy learning :)