>Hi,
>I am trying to run analog to analyse logs coming from a cluster of
>physical machines serving pages for the same web address.
>The whole thing is operated by a layer-4 switch.
>Anybody has any ideas/experience?
>
We run 19 tier two servers (I think, I haven't counted recently,) in
as a server farm under OSF DCE. I think last count was 13 provided
http service via Apache, five are visible to the outside world, so I
have 13 different logs of service covering the (unified,) distributed
file space, (three directory trees.) What happens is I get occasional
duplication warnings, as files are often served nearly simultaneously
from multiple servers, but the counts add up right. Some of our files
are locally replicated, others are central to the core DFS, As far as
Apache is concerned, there is no difference, so my logs are fairly
simple, and pretty much a bare-bones Apache log file.
What I don't know how to do (elegantly,) is differentiate between
servers on a entry by entry basis, as it is a common file space, and
there is no identifying information about which server each entry
came from. I have thought about modifying the Apache log format to do
this, but that is work, and for me this is a pet project, not a real
one. Running 13 reports and then combining results doesn't interest
me either. Fortunately, the load balancing issue is a non-issue for
us, the system admin has that well under control. I am running
reports to satisfy the various bean counters and administrative
types. I use a totally different solution for audience/session
tracking insight.
I simply make a monthly "logs" file, that lists the logs to be
included, that file is then included into the various report
configuration files. For me each node of the DCE that has public HTTP
service has a directory to write it's logs to in the DFS. We don't
run HTTP on our core servers, only on the tier 2 boxes.
># 2000-apr-logs.conf, logs source for analog config files.
># apr 2000
>#
>#
>LOGFORMAT (%S %j %u [%d/%M/%Y:%h:%n:%j %j] "%j %r %j" %c %b)
>LOGFILE /dfs/os/unix/hosts/art/logs/apache-SSL/access_log.20000501.gz
>LOGFILE /dfs/os/unix/hosts/bus/logs/apache-SSL/access_log.20000501.gz
>LOGFILE /dfs/os/unix/hosts/cc/logs/apache-SSL/access_log.20000501.gz
>LOGFILE /dfs/os/unix/hosts/cpc/logs/apache-SSL/access_log.20000501.gz
>LOGFILE /dfs/os/unix/hosts/ec/logs/apache-SSL/access_log.20000501.gz
>LOGFILE /dfs/os/unix/hosts/ent/logs/apache-SSL/access_log.20000501.gz
>LOGFILE /dfs/os/unix/hosts/env/logs/apache-SSL/access_log.20000501.gz
>LOGFILE /dfs/os/unix/hosts/hoc/logs/apache-SSL/access_log.20000501.gz
>LOGFILE /dfs/os/unix/hosts/intranet/logs/apache-SSL/access_log.20000501.gz
>LOGFILE /dfs/os/unix/hosts/itacmv/logs/apache-SSL/access_log.20000501.gz
>LOGFILE /dfs/os/unix/hosts/lib/logs/apache-SSL/access_log.20000501.gz
>LOGFILE /dfs/os/unix/hosts/sci/logs/apache-SSL/access_log.20000501.gz
>LOGFILE /dfs/os/unix/hosts/uob/logs/apache-SSL/access_log.20000501.gz
Then I have a logs.conf that simply consolidates the monthly log
collections in one place, so when I want a report of spanned months,
all I have to do is un-comment various reports. I don't use wild
cards, instead I have a script that builds this file. There are some
files that we exclude from the public reports.
># Logs.conf, logs source for analog config files.
>#
>## CONFIGFILE ./config/1999-jun-logs.conf
>## CONFIGFILE ./config/1999-jul-logs.conf
>## CONFIGFILE ./config/1999-aug-logs.conf
>## CONFIGFILE ./config/1999-sep-logs.conf
>## CONFIGFILE ./config/1999-oct-logs.conf
>## CONFIGFILE ./config/1999-nov-logs.conf
>## CONFIGFILE ./config/1999-dec-logs.conf
>## CONFIGFILE ./config/2000-jan-logs.conf
>## CONFIGFILE ./config/2000-feb-logs.conf
>## CONFIGFILE ./config/2000-mar-logs.conf
>CONFIGFILE ./config/2000-apr-logs.conf
Finally, all this is included into the various reports by yet another include.
>##### LOG DATA #####
>#
>CONFIGFILE ./config/logs.conf
Mail me if you would like more info.
It just keeps the mess under control to have the log files collected
in one place with one script, the time span of a report (which months
logs,) in another file and then the specifics of a report in a third.
It has worked for me quite well.
My next pet project in this respect is to do something about host
lookups, they take forever. I would like to just skip 'em but I have
a feeling the users want them. I should also probably move up to the
current version of Analog.
---
John Ringloff <[EMAIL PROTECTED]>
University Webmaster, Enterprise Computing (V) 909.869.5086
Instructional and Information Technology (F) 909.869.4330
California State Polytechnic University, Pomona <KQ6PX>
------------------------------------------------------------------------
This is the analog-help mailing list. To unsubscribe from this
mailing list, send mail to [EMAIL PROTECTED]
with "unsubscribe" in the main BODY OF THE MESSAGE.
List archived at http://www.mail-archive.com/[email protected]/
------------------------------------------------------------------------
