Hi,
back on this topic a bit.
We have over 20 servers runnig Apache on Linux, and the setup I have used
rotates log files using cronolog, we have 1 log per server (or per virtual
server where needed). So from the httpd.conf:
CustomLog "|/usr/local/sbin/cronolog
/var/log/httpd/www.demasiado.com/%Y/s14-www-access-%m.log" combined
same for error_logs
This makes processing (perl) of the log files (by extracting stuff from
httpd.conf) quite easy... no problems runnig agregate reports either.
All the machines (they're behind a load balancer) NFS export the logs to a
single machine where all the logs are being processed... using analog
(what else?!)
DNS lookups -- I use dnstran, it is great. The only problem I couldn't
solve so far was making Analog use the compressed file that dnstran
produces...
--Josef D.
On Thu, 18 May 2000, John Ringloff wrote:
> >John Ringloff wrote:
> >
> >> What I don't know how to do (elegantly,) is differentiate between
> >> servers on a entry by entry basis, as it is a common file space, and
> >> there is no identifying information about which server each entry
> >> came from.
> >
> >There is in your logfile names:
> >
> >>LOGFILE /dfs/os/unix/hosts/art/logs/apache-SSL/access_log.20000501.gz
> >>LOGFILE /dfs/os/unix/hosts/bus/logs/apache-SSL/access_log.20000501.gz
> >I assume the directory after hosts is the originating server. Since
> >Analog doesn't
> >really do anything with this data (although I suppose you could
> >hijack the Virtual
> >Host report into something like a Servers report), you could just
> >use the names to
> >differentiate host-specific reports, much like you are doing with dates.
>
> Yes, and yes, I just haven't found it worth the effort, The thought I
> had is that I would tag the logs with the machine nameon log entries
> such that I could do something with it later, but analog doesn't have
> a order/count by pre-labled "junk" in the log line routine, That
> would be cool though. Then I could do ONE report, and and along with
> by directory etc, I could do a by host break down of how pages are
> being served.
>
>
> >
> >Running the stats for a 20-server web farm is a pet project? What's
> >you're real
> >job like? :)
>
>
> Usually, I have no more work to do for stats than prepping scripts.
> Analog installed effortlessly, I modified only a few things for
> presentation and have found it a joy to use.
>
> Educational institution: massive projects, impossible deadlines,
> under-compensated/overworked, no resources, and plenty of politics.
> The only thing I can't really tolerate in the above is the politics,
> and the ironic situation that IT folk rarely are able to keep their
> skills current at educational sites due to workload. We use a team
> approach, there are about 20 people who group up as needed that keep
> the all our machines running, including the DCE. I am just the web
> guy, we have a really good system admin, and a CS prof was also
> involved in design/implementation of the DCE. Our system admin has
> done most of the dirty work that makes life good for everyone.
>
> You get to work with interesting people though, and you get to do
> more than the pigeon holes that are now popular in the dot com
> industry. Remember when you could get a job as a "web guy", now your
> a developer, or designer, etc. No variety, little cross training,
> less fun.
>
> One week you're running wire and placing switches, the next we are
> web design, the next system administration. It's great, I just wish
> there was some money in it.
>
> >Anyway, it's fairly easy with Apache Custom Logformat commands to add a server
> >identifier to the logfile entries. You can then use Analog to
> >process this into a
> >vitual host report by asigning it the %v field in the format. Alternately, you
> >could append a 'vitural host' URL to each LOGFILE entry in log confs and use
> >'virtual host' features that way.
>
> I didn't think of using the virtual host feature to handle the
> report, that is good. Thanks! I think I may give that a whirl on the
> test platform.
>
> > > My next pet project in this respect is to do something about host
> >> lookups, they take forever. I would like to just skip 'em but I have
> >> a feeling the users want them. I should also probably move up to the
> > > current version of Analog.
> >
> >Try a third party lookup tool. DSNtran is very fast, especially on
> >Linux. If all
> >your servers are Sun (which is slow at DNS lookup), you might want to throw
> >together a Linux box just to do lookups, with it's own nameserver and a short
> >cycle for rechecks.
>
> SunOS5.6. I have a FreeBSD image sitting on a machine in my office, I
> just haven't done anything with it yet. Thanks I will consider it.
>
> Thanks again,
>
> ---
> John Ringloff <[EMAIL PROTECTED]>
> University Webmaster, Enterprise Computing (V) 909.869.5086
> Instructional and Information Technology (F) 909.869.4330
> California State Polytechnic University, Pomona <KQ6PX>
> ------------------------------------------------------------------------
> This is the analog-help mailing list. To unsubscribe from this
> mailing list, send mail to [EMAIL PROTECTED]
> with "unsubscribe" in the main BODY OF THE MESSAGE.
> List archived at http://www.mail-archive.com/[email protected]/
> ------------------------------------------------------------------------
>
------------------------------------------------------------------------
This is the analog-help mailing list. To unsubscribe from this
mailing list, send mail to [EMAIL PROTECTED]
with "unsubscribe" in the main BODY OF THE MESSAGE.
List archived at http://www.mail-archive.com/[email protected]/
------------------------------------------------------------------------
