Brad,
you can deny DNS lookup, by providing -n option to fw logexport. But most of
the time you
need Names instead of IP addresses in your final report so you need DNS lookups. This
will
become very slow for some lookups and overall increase the time to process logs. One
to
implement this use some sort of caching. (like DNS server (named) also does, but that
is
more complicated. and I won't go there I sometimes back wrote the log analyzer in perl
and
there I use the DBM files to store DNS resolved names:
So everyday you parse your log files and resolve IP addresses and store the resolved IP
address in DBM hash. Next time you encounter the same IP address you simply lookup
locally
stored DBM database, instead of DNS server and wait for long time. So theoritically you
keep these resolved names forever in your disk as DBM hash, but what if somebody change
the name for IP addresses. So what you can do specify kind of TTL(Time to live value)
for
the DBM hash itself and after TTL expired, a new DBM hash would be created and old one
deleted. So let's say if you keep these TTL (1 month, assuming not many IP addresses
(Names) changed during this period,) you won't be resolving same IP addresses each
time
you parse log files. This greatly enhances the conversion speed.
If you are intrested in this, just look through perl code and see how I implemented
this.
http://www.geocities.com/rxknh/pub/fwlogstat/index.html
Rajeev
Brad Grant wrote:
>
> Is there a more efficient way to export logs than with fw logexport.
> Has anyone developed a way to use dbi and a database to perform lookups
> against a database. This is how I parse my apache log files for
> archiving . We are running VPN-1 4.1 on an Ultra 10, and have a full T3
>
> to the internet. I would like to export my logfiles to resolved text
> and compress them each day for permanant archive, but it does not seem
> logistically possible due to the time it takes to export the log files.
> Does fw logexport really perform an nslookup for each entry as it
> appears? Is the data structure of the binary format logfile published?
> Thanks in advance,
>
> Brad Grant
> Network Manager
> Savannah College of Art and Design
> Savannah, GA
> (912) 525-6147
>
> ================================================================================
> To unsubscribe from this mailing list, please see the instructions at
> http://www.checkpoint.com/services/mailing.html
> ================================================================================
--
#########################################################################
(Titanic creators used Linux to simulate the sinking of the great ship)
#########################################################################
Rajeev Kumar ([EMAIL PROTECTED])
Fluent Inc. 10, Cavendish Court, Lebanon NH-03766
-------------------------------------------------------------------------
Phone :: (603)-643-2600 x 349 Fax :: (603)-643-3967
Web:: http://www.fluent.com
#########################################################################
================================================================================
To unsubscribe from this mailing list, please see the instructions at
http://www.checkpoint.com/services/mailing.html
================================================================================
