Re: qmailloganalyzer 0.6
The commands I posted will not erase your logs, my only guess is that you are still doing something wrong. - Original Message - From: [EMAIL PROTECTED] To: "Tim Hunter" [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Wednesday, February 21, 2001 5:07 PM Subject: RE: qmailloganalyzer 0.6 Hi Tim; I tried the changes you suggested and now I've lost another 2 days of logs. I'm not sure why I'm having su much trouble with this program. Anyone else have any suggestions on how to make this work? Still getting gzip complaining about the archive, and still not getting any data being processed into the html files. Once again, is there a way to re-run this on the archived logs? I need a way to get these stats. Thanks again for your help, Tren. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Tren Blackburn - Ownermailto:[EMAIL PROTECTED] = = End of Time Networks http://www.theendoftime.net - - (403) 269-2122 = =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- On Tue, 20 Feb 2001, Tim Hunter wrote: I had similar problems but it turned out I was executing it wrong. I got the following tips from the author: -s and -t usually do not take arguments. make sure that tailocal is in your path If you use daemontools0.70 you could do the following ln -s /source/path/to/tai64nlocal /usr/local/bin/tailocal then run the script with ./qmailloganalyzer.pl -s -t -d ./archive Those parameters were ones I was explicitly trying to use, probably you want different ones. I would think this is the same for you since my errors looked the same and I didn't end up with any data either. -- Tim -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Tuesday, February 20, 2001 11:37 AM To: Jrgen Hoffmann Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: qmailloganalyzer 0.6 Hi; I've tried using this program and must admit am getting frustrated. I've now lost 5 days of logs from the script. I get this error: [root@theendoftime qmail]# ./qmailloganalyzer.pl proceeding [EMAIL PROTECTED] zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file And it proceeds to write web stats with no information. Check http://www.theendoftime.net/maillog. It worked perfectly for the first few days, but now I can't get anything going. Is it possible to re-run this over the gziped logs? Regards, Tren. P.S. I downloaded it from your site again, and that version did not have the modified pipe line reading from file descriptor 5 that you had me check for. Can you please e-mail me the latest version as it doesn't seem to be on your site. Regards, Tren. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Tren Blackburn - Ownermailto:[EMAIL PROTECTED] = = End of Time Networks http://www.theendoftime.net - - (403) 269-2122 = =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- On Thu, 15 Feb 2001, [iso-8859-1] Jrgen Hoffmann wrote: Hi all, I have released qmailloganalyzer 0.6 today. it is available on qmaillog.byteaction.de kind regards Juergen Hoffmann
Re: qmailloganalyzer 0.6
Hi Tren, Yes. I will write it tomorrow and send it to you. Kind regards Juergen Hoffmann On Tue, Feb 20, 2001 at 09:37:13AM -0700, [EMAIL PROTECTED] wrote: Hi; I've tried using this program and must admit am getting frustrated. I've now lost 5 days of logs from the script. I get this error: [root@theendoftime qmail]# ./qmailloganalyzer.pl proceeding [EMAIL PROTECTED] zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file And it proceeds to write web stats with no information. Check http://www.theendoftime.net/maillog. It worked perfectly for the first few days, but now I can't get anything going. Is it possible to re-run this over the gziped logs? Regards, Tren. P.S. I downloaded it from your site again, and that version did not have the modified pipe line reading from file descriptor 5 that you had me check for. Can you please e-mail me the latest version as it doesn't seem to be on your site. Regards, Tren. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Tren Blackburn - Ownermailto:[EMAIL PROTECTED] = = End of Time Networks http://www.theendoftime.net - - (403) 269-2122 = =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- On Thu, 15 Feb 2001, [iso-8859-1] Jürgen Hoffmann wrote: Hi all, I have released qmailloganalyzer 0.6 today. it is available on qmaillog.byteaction.de kind regards Juergen Hoffmann --
RE: qmailloganalyzer 0.6
Hi Tim; I tried the changes you suggested and now I've lost another 2 days of logs. I'm not sure why I'm having su much trouble with this program. Anyone else have any suggestions on how to make this work? Still getting gzip complaining about the archive, and still not getting any data being processed into the html files. Once again, is there a way to re-run this on the archived logs? I need a way to get these stats. Thanks again for your help, Tren. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Tren Blackburn - Ownermailto:[EMAIL PROTECTED] = = End of Time Networks http://www.theendoftime.net - - (403) 269-2122 = =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- On Tue, 20 Feb 2001, Tim Hunter wrote: I had similar problems but it turned out I was executing it wrong. I got the following tips from the author: -s and -t usually do not take arguments. make sure that tailocal is in your path If you use daemontools0.70 you could do the following ln -s /source/path/to/tai64nlocal /usr/local/bin/tailocal then run the script with ./qmailloganalyzer.pl -s -t -d ./archive Those parameters were ones I was explicitly trying to use, probably you want different ones. I would think this is the same for you since my errors looked the same and I didn't end up with any data either. -- Tim -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Tuesday, February 20, 2001 11:37 AM To: Jürgen Hoffmann Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: qmailloganalyzer 0.6 Hi; I've tried using this program and must admit am getting frustrated. I've now lost 5 days of logs from the script. I get this error: [root@theendoftime qmail]# ./qmailloganalyzer.pl proceeding [EMAIL PROTECTED] zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file And it proceeds to write web stats with no information. Check http://www.theendoftime.net/maillog. It worked perfectly for the first few days, but now I can't get anything going. Is it possible to re-run this over the gziped logs? Regards, Tren. P.S. I downloaded it from your site again, and that version did not have the modified pipe line reading from file descriptor 5 that you had me check for. Can you please e-mail me the latest version as it doesn't seem to be on your site. Regards, Tren. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Tren Blackburn - Ownermailto:[EMAIL PROTECTED] = = End of Time Networks http://www.theendoftime.net - - (403) 269-2122 = =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- On Thu, 15 Feb 2001, [iso-8859-1] Jürgen Hoffmann wrote: Hi all, I have released qmailloganalyzer 0.6 today. it is available on qmaillog.byteaction.de kind regards Juergen Hoffmann
Re: qmailloganalyzer 0.6
Hi; I've tried using this program and must admit am getting frustrated. I've now lost 5 days of logs from the script. I get this error: [root@theendoftime qmail]# ./qmailloganalyzer.pl proceeding [EMAIL PROTECTED] zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file And it proceeds to write web stats with no information. Check http://www.theendoftime.net/maillog. It worked perfectly for the first few days, but now I can't get anything going. Is it possible to re-run this over the gziped logs? Regards, Tren. P.S. I downloaded it from your site again, and that version did not have the modified pipe line reading from file descriptor 5 that you had me check for. Can you please e-mail me the latest version as it doesn't seem to be on your site. Regards, Tren. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Tren Blackburn - Ownermailto:[EMAIL PROTECTED] = = End of Time Networks http://www.theendoftime.net - - (403) 269-2122 = =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- On Thu, 15 Feb 2001, [iso-8859-1] Jürgen Hoffmann wrote: Hi all, I have released qmailloganalyzer 0.6 today. it is available on qmaillog.byteaction.de kind regards Juergen Hoffmann
RE: qmailloganalyzer 0.6
I had similar problems but it turned out I was executing it wrong. I got the following tips from the author: -s and -t usually do not take arguments. make sure that tailocal is in your path If you use daemontools0.70 you could do the following ln -s /source/path/to/tai64nlocal /usr/local/bin/tailocal then run the script with ./qmailloganalyzer.pl -s -t -d ./archive Those parameters were ones I was explicitly trying to use, probably you want different ones. I would think this is the same for you since my errors looked the same and I didn't end up with any data either. -- Tim -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Tuesday, February 20, 2001 11:37 AM To: Jrgen Hoffmann Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: qmailloganalyzer 0.6 Hi; I've tried using this program and must admit am getting frustrated. I've now lost 5 days of logs from the script. I get this error: [root@theendoftime qmail]# ./qmailloganalyzer.pl proceeding [EMAIL PROTECTED] zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file zcat: ./qmail-20010220.gz: unexpected end of file And it proceeds to write web stats with no information. Check http://www.theendoftime.net/maillog. It worked perfectly for the first few days, but now I can't get anything going. Is it possible to re-run this over the gziped logs? Regards, Tren. P.S. I downloaded it from your site again, and that version did not have the modified pipe line reading from file descriptor 5 that you had me check for. Can you please e-mail me the latest version as it doesn't seem to be on your site. Regards, Tren. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Tren Blackburn - Ownermailto:[EMAIL PROTECTED] = = End of Time Networks http://www.theendoftime.net - - (403) 269-2122 = =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- On Thu, 15 Feb 2001, [iso-8859-1] Jrgen Hoffmann wrote: Hi all, I have released qmailloganalyzer 0.6 today. it is available on qmaillog.byteaction.de kind regards Juergen Hoffmann
Re: qmailloganalyzer
Hi Jergen; I have installed your package, however, when I try to run the qmailloganalyser.pl, I get this: [root@theendoftime cgi-bin]# ./qmailloganalyzer.pl proceeding [EMAIL PROTECTED] matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open And all it does it write a bunch of these files (which are zero byte if I un gzip them as you can see) -rw-r--r--1 root www20 Feb 14 06:29 qmail-4000389603587. gz -rw-r--r--1 root www20 Feb 14 06:29 qmail-40003896038594 13.gz -rw-r--r--1 root www20 Feb 14 06:29 qmail-40003896038597 4.gz -rw-r--r--1 root www20 Feb 14 06:29 qmail-40003896038598 5.gz -rw-r--r--1 root www 0 Feb 14 06:29 qmail-40003896160604 24 Any ideas what's going on? Cheers, Tren. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Tren Blackburn - Ownermailto:[EMAIL PROTECTED] = = End of Time Networks http://www.theendoftime.net - - (403) 269-2122 = =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- On Fri, 9 Feb 2001, [iso-8859-1] Jürgen Hoffmann wrote: To all who wanted to try the software, I have set up an explanatory website, that should make the working of the script clear. It can be found at: http://qmaillog.byteaction.de I have included all the bugfixes that were submitted yesterday. So enjoy kind regards Juergen Hoffmann
RE: qmailloganalyzer
I like the pretty graphs. Matt is your script GPL? I have been debating on writing something similiar ( in all my copious free time :) ) , but if yor script is available it would save me alot of time. The only difference is I would rather go striaght to RRDTool rather than through cricket. Thanks, Joe Modjeski [EMAIL PROTECTED] -Original Message- From: Matt Simerson [mailto:[EMAIL PROTECTED]] Sent: Thursday, February 08, 2001 6:26 PM To: '[EMAIL PROTECTED]' Subject: RE: qmailloganalyzer That is pretty cool. I have something cool too. :-) I have a perl script that collects all the files, does some processing on them, feeds the results into cricket (for pretty graphs), and then gzips and archives the log files away. http://admin.mail.safeserver.com/admin/cricket/mail.html Ignore the spikes. Matt -Original Message- From: Jrgen Hoffmann [mailto:[EMAIL PROTECTED]] Sent: Thursday, February 08, 2001 12:47 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: qmailloganalyzer Hi, thanx. multilog is a program that comes with the daemontools package. You can download it from http://www.qmail.org The best way on how to install it iat documented in my life with qmail. If you use daemontools already (supervise, svc, etc.) then you most likely just have to edit your logging options. kind regards Juergen Hoffmann - Original Message - From: [EMAIL PROTECTED] To: "Jrgen Hoffmann" [EMAIL PROTECTED] Sent: Thursday, February 08, 2001 9:10 PM Subject: Re: qmailloganalyzer Hi! Looked at this. What is multilog? The page looks AWESOME and I think I wanna try it. =) Cheers, Tren. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Tren Blackburn - Ownermailto:[EMAIL PROTECTED] = = End of Time Networks http://www.theendoftime.net - - (403) 269-2122 = =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- On Thu, 8 Feb 2001, [iso-8859-1] Jrgen Hoffmann wrote: Hi all, I know this might not be the right list, but since I am using multilog with qmail + vpopmail and I could not find anything that would do good logparsing I thought that I might be best of writing my own perl script which utilizes qmailanalog to its full extent. The Script takes a logfile with tai format converts it to tailocal to determine the correct date. Afterwards pipes it back through localtai to get it back back to the taiformat that qmailanalog needs. Then passes it through matchup and gzips all the files to a directory of your liking for archiving. Then it uses all these archived logfiles and pipes them through the various qmailanalog scripts zoverall, senders, etc. and outputs the result to a webdir of your liking. It saves the result in static html pages in a hierarchichal structure, so you get can view your results over the periods of time. command line options to the script are: b - basename this is the prefix of the lognames if you don't like qmail set it different (qmail-mmdd.gz is default) d - archive directory (defaults to ".") s - do you want to save the source logs (default is "no") t - pipe the logs through tailocal before parsing (default is "no") z - compression (look at the source code (sorry) defaults to standard .gz) m - path to qmailanalog binary files (defaults to /usr/local/qmailanalog/bin) o - outputdirectory (defaults to /var/www/maillog) You can see an example of the output at http://www.web-4-all.de/maillog AGAIN This is only for multilog. Please let me know if this software is useful to you and don't hesitate to contact me if you have any questions. I will post the script to the qmail mailing list later on, or maybe one of you guys can do it for me, since I am really busy at the moment. I will setup a website also which will be available via http://qmaillog.byteaction.de by the end of the month. Planned features include graphs, Column Totals where column totals ought to be. Any other feature requests? Ken feel free to distribute this with vpopmail if you want to. kind regards Juergen Hoffmann P.S. the qmailloganalyzer.css has to be copied to the root directory of your webserver for it to work. If you do not use it the results will look crabby.
RE: qmailloganalyzer
About this problem. /var/log/qmail with multilog is rotated alotwith so much email being sent all the time...how acurate will the script be being alot is lost after rotation? On Tue, 13 Feb 2001, Joe Modjeski wrote: Date: Tue, 13 Feb 2001 12:33:10 -0700 From: Joe Modjeski [EMAIL PROTECTED] To: Matt Simerson [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: RE: qmailloganalyzer I like the pretty graphs. Matt is your script GPL? I have been debating on writing something similiar ( in all my copious free time :) ) , but if yor script is available it would save me alot of time. The only difference is I would rather go striaght to RRDTool rather than through cricket. Thanks, Joe Modjeski [EMAIL PROTECTED] -Original Message- From: Matt Simerson [mailto:[EMAIL PROTECTED]] Sent: Thursday, February 08, 2001 6:26 PM To: '[EMAIL PROTECTED]' Subject: RE: qmailloganalyzer That is pretty cool. I have something cool too. :-) I have a perl script that collects all the files, does some processing on them, feeds the results into cricket (for pretty graphs), and then gzips and archives the log files away. http://admin.mail.safeserver.com/admin/cricket/mail.html Ignore the spikes. Matt -Original Message- From: Jürgen Hoffmann [mailto:[EMAIL PROTECTED]] Sent: Thursday, February 08, 2001 12:47 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: qmailloganalyzer Hi, thanx. multilog is a program that comes with the daemontools package. You can download it from http://www.qmail.org The best way on how to install it iat documented in my life with qmail. If you use daemontools already (supervise, svc, etc.) then you most likely just have to edit your logging options. kind regards Juergen Hoffmann - Original Message - From: [EMAIL PROTECTED] To: "Jürgen Hoffmann" [EMAIL PROTECTED] Sent: Thursday, February 08, 2001 9:10 PM Subject: Re: qmailloganalyzer Hi! Looked at this. What is multilog? The page looks AWESOME and I think I wanna try it. =) Cheers, Tren. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Tren Blackburn - Ownermailto:[EMAIL PROTECTED] = = End of Time Networks http://www.theendoftime.net - - (403) 269-2122 = =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- On Thu, 8 Feb 2001, [iso-8859-1] Jürgen Hoffmann wrote: Hi all, I know this might not be the right list, but since I am using multilog with qmail + vpopmail and I could not find anything that would do good logparsing I thought that I might be best of writing my own perl script which utilizes qmailanalog to its full extent. The Script takes a logfile with tai format converts it to tailocal to determine the correct date. Afterwards pipes it back through localtai to get it back back to the taiformat that qmailanalog needs. Then passes it through matchup and gzips all the files to a directory of your liking for archiving. Then it uses all these archived logfiles and pipes them through the various qmailanalog scripts zoverall, senders, etc. and outputs the result to a webdir of your liking. It saves the result in static html pages in a hierarchichal structure, so you get can view your results over the periods of time. command line options to the script are: b - basename this is the prefix of the lognames if you don't like qmail set it different (qmail-mmdd.gz is default) d - archive directory (defaults to ".") s - do you want to save the source logs (default is "no") t - pipe the logs through tailocal before parsing (default is "no") z - compression (look at the source code (sorry) defaults to standard .gz) m - path to qmailanalog binary files (defaults to /usr/local/qmailanalog/bin) o - outputdirectory (defaults to /var/www/maillog) You can see an example of the output at http://www.web-4-all.de/maillog AGAIN This is only for multilog. Please let me know if this software is useful to you and don't hesitate to contact me if you have any questions. I will post the script to the qmail mailing list later on, or maybe one of you guys can do it for me, since I am really busy at the moment. I will setup a website also which will be available via http://qmaillog.byteaction.de by the end of the month. Planned features include graphs, Column Totals where column totals ought to be. Any other feature requests? Ken feel free to distribute this with vpopmail if you want to. kind regards Juergen Hoffmann P.S. the qmailloganalyzer.css has to be copied to the root directory of your webserver for it to work. If you do not use it the results will look crabby.
RE: qmailloganalyzer
You know what you really need you need to crontab this shit like every min or 5 min for large traffic where thos /var/log/qmail files change so oftenshould have a crontab option. On Tue, 13 Feb 2001, Dan Phoenix wrote: Date: Tue, 13 Feb 2001 11:55:21 -0800 (PST) From: Dan Phoenix [EMAIL PROTECTED] To: Joe Modjeski [EMAIL PROTECTED] Cc: Matt Simerson [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: RE: qmailloganalyzer About this problem. /var/log/qmail with multilog is rotated alotwith so much email being sent all the time...how acurate will the script be being alot is lost after rotation? On Tue, 13 Feb 2001, Joe Modjeski wrote: Date: Tue, 13 Feb 2001 12:33:10 -0700 From: Joe Modjeski [EMAIL PROTECTED] To: Matt Simerson [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: RE: qmailloganalyzer I like the pretty graphs. Matt is your script GPL? I have been debating on writing something similiar ( in all my copious free time :) ) , but if yor script is available it would save me alot of time. The only difference is I would rather go striaght to RRDTool rather than through cricket. Thanks, Joe Modjeski [EMAIL PROTECTED] -Original Message- From: Matt Simerson [mailto:[EMAIL PROTECTED]] Sent: Thursday, February 08, 2001 6:26 PM To: '[EMAIL PROTECTED]' Subject: RE: qmailloganalyzer That is pretty cool. I have something cool too. :-) I have a perl script that collects all the files, does some processing on them, feeds the results into cricket (for pretty graphs), and then gzips and archives the log files away. http://admin.mail.safeserver.com/admin/cricket/mail.html Ignore the spikes. Matt -Original Message- From: Jürgen Hoffmann [mailto:[EMAIL PROTECTED]] Sent: Thursday, February 08, 2001 12:47 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: qmailloganalyzer Hi, thanx. multilog is a program that comes with the daemontools package. You can download it from http://www.qmail.org The best way on how to install it iat documented in my life with qmail. If you use daemontools already (supervise, svc, etc.) then you most likely just have to edit your logging options. kind regards Juergen Hoffmann - Original Message - From: [EMAIL PROTECTED] To: "Jürgen Hoffmann" [EMAIL PROTECTED] Sent: Thursday, February 08, 2001 9:10 PM Subject: Re: qmailloganalyzer Hi! Looked at this. What is multilog? The page looks AWESOME and I think I wanna try it. =) Cheers, Tren. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Tren Blackburn - Ownermailto:[EMAIL PROTECTED] = = End of Time Networks http://www.theendoftime.net - - (403) 269-2122 = =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- On Thu, 8 Feb 2001, [iso-8859-1] Jürgen Hoffmann wrote: Hi all, I know this might not be the right list, but since I am using multilog with qmail + vpopmail and I could not find anything that would do good logparsing I thought that I might be best of writing my own perl script which utilizes qmailanalog to its full extent. The Script takes a logfile with tai format converts it to tailocal to determine the correct date. Afterwards pipes it back through localtai to get it back back to the taiformat that qmailanalog needs. Then passes it through matchup and gzips all the files to a directory of your liking for archiving. Then it uses all these archived logfiles and pipes them through the various qmailanalog scripts zoverall, senders, etc. and outputs the result to a webdir of your liking. It saves the result in static html pages in a hierarchichal structure, so you get can view your results over the periods of time. command line options to the script are: b - basename this is the prefix of the lognames if you don't like qmail set it different (qmail-mmdd.gz is default) d - archive directory (defaults to ".") s - do you want to save the source logs (default is "no") t - pipe the logs through tailocal before parsing (default is "no") z - compression (look at the source code (sorry) defaults to standard .gz) m - path to qmailanalog binary files (defaults to /usr/local/qmailanalog/bin) o - outputdirectory (defaults to /var/www/maillog) You can see an example of the output at http://www.web-4-all.de/maillog AGAIN This is only for multilog. Please let me know if this software is useful to you and don't hesitate to contact me if you have any questions. I will post the script to the qmail ma
Re: qmailloganalyzer
You know what you really need you need to crontab this shit like every min or 5 min for large traffic where thos /var/log/qmail files change so oftenshould have a crontab option. It is called crontab -e Setting up crontab tasks for a script like this is the job of the system admin, and not the software developer.
Re: qmailloganalyzer
Hi, thanx. multilog is a program that comes with the daemontools package. You can download it from http://www.qmail.org The best way on how to install it iat documented in my life with qmail. If you use daemontools already (supervise, svc, etc.) then you most likely just have to edit your logging options. kind regards Juergen Hoffmann - Original Message - From: [EMAIL PROTECTED] To: "Jrgen Hoffmann" [EMAIL PROTECTED] Sent: Thursday, February 08, 2001 9:10 PM Subject: Re: qmailloganalyzer Hi! Looked at this. What is multilog? The page looks AWESOME and I think I wanna try it. =) Cheers, Tren. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Tren Blackburn - Ownermailto:[EMAIL PROTECTED] = = End of Time Networks http://www.theendoftime.net - - (403) 269-2122 = =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- On Thu, 8 Feb 2001, [iso-8859-1] Jrgen Hoffmann wrote: Hi all, I know this might not be the right list, but since I am using multilog with qmail + vpopmail and I could not find anything that would do good logparsing I thought that I might be best of writing my own perl script which utilizes qmailanalog to its full extent. The Script takes a logfile with tai format converts it to tailocal to determine the correct date. Afterwards pipes it back through localtai to get it back back to the taiformat that qmailanalog needs. Then passes it through matchup and gzips all the files to a directory of your liking for archiving. Then it uses all these archived logfiles and pipes them through the various qmailanalog scripts zoverall, senders, etc. and outputs the result to a webdir of your liking. It saves the result in static html pages in a hierarchichal structure, so you get can view your results over the periods of time. command line options to the script are: b - basename this is the prefix of the lognames if you don't like qmail set it different (qmail-mmdd.gz is default) d - archive directory (defaults to ".") s - do you want to save the source logs (default is "no") t - pipe the logs through tailocal before parsing (default is "no") z - compression (look at the source code (sorry) defaults to standard .gz) m - path to qmailanalog binary files (defaults to /usr/local/qmailanalog/bin) o - outputdirectory (defaults to /var/www/maillog) You can see an example of the output at http://www.web-4-all.de/maillog AGAIN This is only for multilog. Please let me know if this software is useful to you and don't hesitate to contact me if you have any questions. I will post the script to the qmail mailing list later on, or maybe one of you guys can do it for me, since I am really busy at the moment. I will setup a website also which will be available via http://qmaillog.byteaction.de by the end of the month. Planned features include graphs, Column Totals where column totals ought to be. Any other feature requests? Ken feel free to distribute this with vpopmail if you want to. kind regards Juergen Hoffmann P.S. the qmailloganalyzer.css has to be copied to the root directory of your webserver for it to work. If you do not use it the results will look crabby.
RE: qmailloganalyzer
Because we all use qmail and most of us like a pretty useful way to view stats and logs. Don't flame the guy, I applaud him. I don't know how many times I have heard complaints about not having a good log analyzer that works with vpopmail, or even qmail at all for that matter. -Original Message- From: Kari Suomela [mailto:[EMAIL PROTECTED]] Sent: Thursday, February 08, 2001 4:33 PM To: [EMAIL PROTECTED] Subject: qmailloganalyzer Thursday February 08 2001 19:51, =?iso-8859-1?Q?J=FCrgen_Hoffmann?= wrote to All: iH This is a multi-part message in MIME format. iH Hi all, iH I know this might not be the right list, but since I am using iH multilog So why post this crap!? KS
Re: qmailloganalyzer
Hi, I will write a detailed README tomorrow. Plus I will add a -h flag to print out the usage of the script. kind regards Juergen Hoffmann - Original Message - From: "Shao Ming" [EMAIL PROTECTED] To: "Jrgen Hoffmann" [EMAIL PROTECTED] Sent: Thursday, February 08, 2001 11:47 PM Subject: Re: qmailloganalyzer Hi! I want to give your package a try but can you attach some 'README' to help me get started? cheers! - Original Message - From: "Jrgen Hoffmann" [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Friday, February 09, 2001 4:46 AM Subject: Re: qmailloganalyzer Hi, thanx. multilog is a program that comes with the daemontools package. You can download it from http://www.qmail.org The best way on how to install it iat documented in my life with qmail. If you use daemontools already (supervise, svc, etc.) then you most likely just have to edit your logging options. kind regards Juergen Hoffmann - Original Message - From: [EMAIL PROTECTED] To: "Jrgen Hoffmann" [EMAIL PROTECTED] Sent: Thursday, February 08, 2001 9:10 PM Subject: Re: qmailloganalyzer Hi! Looked at this. What is multilog? The page looks AWESOME and I think I wanna try it. =) Cheers, Tren. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - Tren Blackburn - Ownermailto:[EMAIL PROTECTED] = = End of Time Networks http://www.theendoftime.net - - (403) 269-2122 = =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- On Thu, 8 Feb 2001, [iso-8859-1] Jrgen Hoffmann wrote: Hi all, I know this might not be the right list, but since I am using multilog with qmail + vpopmail and I could not find anything that would do good logparsing I thought that I might be best of writing my own perl script which utilizes qmailanalog to its full extent. The Script takes a logfile with tai format converts it to tailocal to determine the correct date. Afterwards pipes it back through localtai to get it back back to the taiformat that qmailanalog needs. Then passes it through matchup and gzips all the files to a directory of your liking for archiving. Then it uses all these archived logfiles and pipes them through the various qmailanalog scripts zoverall, senders, etc. and outputs the result to a webdir of your liking. It saves the result in static html pages in a hierarchichal structure, so you get can view your results over the periods of time. command line options to the script are: b - basename this is the prefix of the lognames if you don't like qmail set it different (qmail-mmdd.gz is default) d - archive directory (defaults to ".") s - do you want to save the source logs (default is "no") t - pipe the logs through tailocal before parsing (default is "no") z - compression (look at the source code (sorry) defaults to standard .gz) m - path to qmailanalog binary files (defaults to /usr/local/qmailanalog/bin) o - outputdirectory (defaults to /var/www/maillog) You can see an example of the output at http://www.web-4-all.de/maillog AGAIN This is only for multilog. Please let me know if this software is useful to you and don't hesitate to contact me if you have any questions. I will post the script to the qmail mailing list later on, or maybe one of you guys can do it for me, since I am really busy at the moment. I will setup a website also which will be available via http://qmaillog.byteaction.de by the end of the month. Planned features include graphs, Column Totals where column totals ought to be. Any other feature requests? Ken feel free to distribute this with vpopmail if you want to. kind regards Juergen Hoffmann P.S. the qmailloganalyzer.css has to be copied to the root directory of your webserver for it to work. If you do not use it the results will look crabby.
Re: qmailloganalyzer
I know this might not be the right list, but since I am using multilog with qmail + vpopmail and I could not find anything that would do good logparsing I thought that I might be best of writing my own perl script which utilizes qmailanalog to its full extent. Interesting that this has come up as its something ive been trying to do for ages now, and given up. I have qmailanalog installed, multilog doing the logging, and vpopmail. Imtrying to get a good log analysis program working, but all my logfiles are in the form @timestamp.s like @40003a6b6eac32a52fe4.s and no log analyser seems to be able to decode them or make any sense, but then again its possible im just not using them properly, any advice on this would be great!
Re: qmailloganalyzer
Hi, make sure that if you are using multilog that you pass the argument -t This is needed, because the logs timestamps are in tai format (e.g. @4003829some odd number) -t makes sure that these numbers are first converted to localtime Format So the Script can determine the day. The Script uses the command tailocal. If this is named tai64nlocal on your machine make a symlink or edit the code. after the script has determined the date of the logentry it pipes it through localtai, and converts it to a timestring that is suitable for qmailanalogs matchup. Besides by after converting it gets passed through matchup right away and is then packed into a gzipped archive. And by the way make sure that you submit the -s switch. There is a bug in the code close to unlink, which won't delete the old logfiles, but wil in future releases. The -s switch will make sure that the logs stay in place. After that you should be set. Hope that helps. Let me know. kind regards Juergen Hoffmann - Original Message - From: "Shao Ming" [EMAIL PROTECTED] To: "Jrgen Hoffmann" [EMAIL PROTECTED] Sent: Friday, February 09, 2001 12:24 AM Subject: Re: qmailloganalyzer Hi! first of all, I did make another change to your script. The script cough out the following error sh: localtai: command not found so I make a change to line 74 $localtai = "./localtai"; I ran the script without any arguments. The system is Redhat Linux 6.2 and the perl version is as below ... # perl -v This is perl, version 5.005_03 built for i386-linux Copyright 1987-1999, Larry Wall Perl may be copied only under the terms of either the Artistic License or the GNU General Public License, which may be found in the Perl 5.0 source kit. Complete documentation for Perl, including FAQ lists, should be found on this system using `man perl' or `perldoc perl'. If you have access to the Internet, point your browser at http://www.perl.com/, the Perl Home Page. The scripts did create lots of qmail-xxx.gz logs around in the same directory. cheers! - Original Message - From: "Jrgen Hoffmann" [EMAIL PROTECTED] To: "Shao Ming" [EMAIL PROTECTED] Sent: Friday, February 09, 2001 7:07 AM Subject: Re: qmailloganalyzer Hi, what arguments did you give to the script? match writes out pending message on Dsecriptor five. I have not found a way to solve this. What Perl Version are you using? kind regards Juergen Hoffmann - Original Message - From: "Shao Ming" [EMAIL PROTECTED] To: "Jrgen Hoffmann" [EMAIL PROTECTED] Sent: Friday, February 09, 2001 12:05 AM Subject: Re: qmailloganalyzer Here some error I have encountered in my initial run. Not enough arguments for mkdir at ./qmailloganalyzer.pl line 78, near "$outputdir }" Not enough arguments for mkdir at ./qmailloganalyzer.pl line 182, near ""$outputdir/$year" }" Not enough arguments for mkdir at ./qmailloganalyzer.pl line 183, near ""$outputdir/$year/$month" }" Not enough arguments for mkdir at ./qmailloganalyzer.pl line 184, near ""$outputdir/$year/$month/$day" }" Not enough arguments for mkdir at ./qmailloganalyzer.pl line 222, near ""$outputdir/$year" }" Not enough arguments for mkdir at ./qmailloganalyzer.pl line 223, near ""$output} so I append the MODE 0777 behind the parameter. Now it run but give me the following error ... matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open matchup: fatal: unable to write fd 5: file descriptor not open had to CTRL-C to get out of the endless loop Hope the above help ... cheers! - Original Message - From: "Jrgen Hoffmann" [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Friday, February 09, 2001 2:51 AM Subject: qmaillogana
Re: qmailloganalyzer
Hi, you could of course use my loganalyzer which is still alpha but gives out good results. If u pipe your logfiles through tai64nlocal you wil get the standard date and fractional seconds behind that. Or is that the process id? I dont really remember. Anyways the usage for tai64local would be: tai64nlocal @214382hfdjhkl.s Your_parsed_log_with_dates.txt kind regards Juergen Hoffmann - Original Message - From: "Daniel Hardaker" [EMAIL PROTECTED] To: "Jrgen Hoffmann" [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Friday, February 09, 2001 12:22 AM Subject: Re: qmailloganalyzer I know this might not be the right list, but since I am using multilog with qmail + vpopmail and I could not find anything that would do good logparsing I thought that I might be best of writing my own perl script which utilizes qmailanalog to its full extent. Interesting that this has come up as its something ive been trying to do for ages now, and given up. I have qmailanalog installed, multilog doing the logging, and vpopmail. Imtrying to get a good log analysis program working, but all my logfiles are in the form @timestamp.s like @40003a6b6eac32a52fe4.s and no log analyser seems to be able to decode them or make any sense, but then again its possible im just not using them properly, any advice on this would be great!
