[PHP] Log parsing
I have a dynamic IP address, I used everydns.net as a DNS server. I have a bash script that runs with cron that checks the current IP of the machine with the DNS record so that if my IP changes; it can run the update program. I would like to do some thorough availability reports, so I started by writing the result of the bash script to a log file. I want to parse the log file to give me availability reports. The log file is in the form: 08/14/04 09:10:01 [TAB] S - when the IP check returns OK 08/14/04 09:10:01 [TAB] U - when the IP need to be synchronised The process does not run when the computer is not on, so that needs to be taken into consideration. Can anyone come up with a script that will carry out this process efficiently? Thanks in advance, Watty
Re: [PHP] Log parsing
Watty wrote: I have a dynamic IP address, I used everydns.net as a DNS server. I have a bash script that runs with cron that checks the current IP of the machine with the DNS record so that if my IP changes; it can run the update program. I would like to do some thorough availability reports, so I started by writing the result of the bash script to a log file. I want to parse the log file to give me availability reports. The log file is in the form: 08/14/04 09:10:01 [TAB] S - when the IP check returns OK 08/14/04 09:10:01 [TAB] U - when the IP need to be synchronised The process does not run when the computer is not on, so that needs to be taken into consideration. Can anyone come up with a script that will carry out this process efficiently? How exactly do you want it parsed? uptime vs. downtime? number of updates? average time between updates? how many zeros are in your file? -- ---John Holmes... Amazon Wishlist: www.amazon.com/o/registry/3BEXC84AB3A5E/ php|architect: The Magazine for PHP Professionals www.phparch.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Log parsing
Watty wrote: I would like to do some thorough availability reports, so I started by writing the result of the bash script to a log file. I want to parse the log file to give me availability reports. The log file is in the form: 08/14/04 09:10:01 [TAB] S - when the IP check returns OK 08/14/04 09:10:01 [TAB] U - when the IP need to be synchronised The process does not run when the computer is not on, so that needs to be taken into consideration. Is there any entry for when the script/computer is started or stopped? If not, you're not going to be able to determine the actual uptime. If one entry is S, then two hours pass and an entry is U, how do we know if the computer was on or off during that time? Basically how this is going to work: 1. Read first two lines, $line[1], $line[2], using fgets() 2. Use strtotime() to convert timestamps and subtract to determine the elapsed time between them 3. Add difference to either $U_time or $S_time based upon status (S or U) in $line[1] 4. Set $line[1] = $line[2] 5. Read next line of file into $line[2] 6. GOTO #2 Then you'll have total $U_time and total $S_time and you can calculate your percentages from there. -- ---John Holmes... Amazon Wishlist: www.amazon.com/o/registry/3BEXC84AB3A5E/ php|architect: The Magazine for PHP Professionals www.phparch.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] Log parsing
We know because the IP check run every 5 minutes by cron, and if it has no run within 5:01 minutes then the computer is off, or the script isn't working. But we should assume that the computer is off is there is not a record for that 5 minute slot. Alternatively we could write another line into the log when the computer shuts down. Watty -Original Message- From: John Holmes [mailto:[EMAIL PROTECTED] Sent: 15 August 2004 01:52 To: Watty Cc: [EMAIL PROTECTED] Subject: Re: [PHP] Log parsing Watty wrote: I would like to do some thorough availability reports, so I started by writing the result of the bash script to a log file. I want to parse the log file to give me availability reports. The log file is in the form: 08/14/04 09:10:01 [TAB] S - when the IP check returns OK 08/14/04 09:10:01 [TAB] U - when the IP need to be synchronised The process does not run when the computer is not on, so that needs to be taken into consideration. Is there any entry for when the script/computer is started or stopped? If not, you're not going to be able to determine the actual uptime. If one entry is S, then two hours pass and an entry is U, how do we know if the computer was on or off during that time? Basically how this is going to work: 1. Read first two lines, $line[1], $line[2], using fgets() 2. Use strtotime() to convert timestamps and subtract to determine the elapsed time between them 3. Add difference to either $U_time or $S_time based upon status (S or U) in $line[1] 4. Set $line[1] = $line[2] 5. Read next line of file into $line[2] 6. GOTO #2 Then you'll have total $U_time and total $S_time and you can calculate your percentages from there. -- ---John Holmes... Amazon Wishlist: www.amazon.com/o/registry/3BEXC84AB3A5E/ php|architect: The Magazine for PHP Professionals - www.phparch.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Log parsing
John Holmes wrote: Watty wrote: We know because the IP check run every 5 minutes by cron, and if it has no run within 5:01 minutes then the computer is off, or the script isn't working. But we should assume that the computer is off is there is not a record for that 5 minute slot. Alternatively we could write another line into the log when the computer shuts down. Basically how this is going to work: 1. Read first two lines, $line[1], $line[2], using fgets() 2. Use strtotime() to convert timestamps and subtract to determine the elapsed time between them 3. Add difference to either $U_time or $S_time based upon status (S or U) in $line[1] 4. Set $line[1] = $line[2] 5. Read next line of file into $line[2] 6. GOTO #2 Then you'll have total $U_time and total $S_time and you can calculate your percentages from there. So, according to what you said, when you calculate the difference in Step 2 above, you could put a maximum of five minutes, right? If the difference between two lines is over 5 minutes, you can assume it was on for the first 5 and off for the rest. -- ---John Holmes... Amazon Wishlist: www.amazon.com/o/registry/3BEXC84AB3A5E/ php|architect: The Magazine for PHP Professionals www.phparch.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] Log parsing
I have got a messy script working, thanks for the help John. It's attached if anyone would mind helping me hack it up. There is also a copy of a sample log file. Watty -Original Message- From: John Holmes [mailto:[EMAIL PROTECTED] Sent: 15 August 2004 01:52 To: Watty Cc: [EMAIL PROTECTED] Subject: Re: [PHP] Log parsing Watty wrote: I would like to do some thorough availability reports, so I started by writing the result of the bash script to a log file. I want to parse the log file to give me availability reports. The log file is in the form: 08/14/04 09:10:01 [TAB] S - when the IP check returns OK 08/14/04 09:10:01 [TAB] U - when the IP need to be synchronised The process does not run when the computer is not on, so that needs to be taken into consideration. Is there any entry for when the script/computer is started or stopped? If not, you're not going to be able to determine the actual uptime. If one entry is S, then two hours pass and an entry is U, how do we know if the computer was on or off during that time? Basically how this is going to work: 1. Read first two lines, $line[1], $line[2], using fgets() 2. Use strtotime() to convert timestamps and subtract to determine the elapsed time between them 3. Add difference to either $U_time or $S_time based upon status (S or U) in $line[1] 4. Set $line[1] = $line[2] 5. Read next line of file into $line[2] 6. GOTO #2 Then you'll have total $U_time and total $S_time and you can calculate your percentages from there. -- ---John Holmes... Amazon Wishlist: www.amazon.com/o/registry/3BEXC84AB3A5E/ php|architect: The Magazine for PHP Professionals - www.phparch.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] Log parsing
I'll try that in another way; feel free to give me your hacks for it:
?php
$file = file_get_contents( ./ip );
$line = explode( \n, $file );
## Number of log entries
$total = 0;
$unavail = 0;
$unavaillog = FALSE;
## Establish the standard difference between log entries of 5 minutes
$diff = 300;
## Add 5 because cron often is off by a couple of seconds
$maxdiff = $diff + 5;
$downtime = array(array());
Done the log parser:
## Count variable for iterations of downtime
$cntdown = 0;
foreach( $line as $val ) {
$bits = explode( \t, trim( $val ) );
if ( $bits[0] ( $bits[1] == S || $bits[1] == U ) ) {
$ctime = $bits[0];
if ( $total == 0 ) {
$firsttime = $ctime;
$ptime = $firsttime;
}
$diftime = $ctime - $ptime;
## Calculate the amount of time the server has
been down.
if( $diftime $maxdiff ) {
print $diftime .\n;
$missed = $diftime / $diff;
$unavail += $missed;
$total += $missed;
if ( $unavailog ) { }
else {
$downtime[$cntdown][start] = $ptime;
$unavailog = TRUE;
}
}
if( trim( $bits[1] ) == U ) {
if ( $unavailog ) {
}
else {
$downtime[$cntdown][start] = $ctime;
$unavailog = TRUE;
}
$unavail++;
}
else {
## Once the down time has finished
if ( $unavailog ) {
$downtime[$cntdown][finish] = $ctime;
$downtime[$cntdown][diff] = $downtime[$cntdown][finish] -
$downtime[$cntdown][start];
$cntdown++;
}
$unavailog = FALSE;
}
$total++;
$ptime = $ctime;
}
}
## Last time is registered as the timestamp of the last entry
$lasttime = $ctime;
$firsttime = date( j/n/y H:i, $firsttime );
$lasttime = date( j/n/y H:i, $lasttime );
## Make sure the percentage is uptime, one decimal place
$percentage = number_format( ( 100 - ( ( $unavail / $total ) * 100 ) ),
1 );
print $percentage% uptime from $firsttime to $lasttime\n;
## Need to make this look pretty
print_r( $downtime );
?
example log file:
1092528602 S
1092528901 S
1092529201 S
1092529502 S
1092530102 S
1092530401 S
1092530701 S
1092531601 S
1092531902 S
1092532201 S
1092532501 U
