Hi folks, I am trying to assess just how chatty Lustre installations can be with logging on systems "in the wild".
I have attached a small bash script which when run against the text file you have your Lustre logs going to (i.e. typically via syslog) counts the number of log entries[1], the average number of entries per second over the time period the log covers and a peak number of records for any given second. It's output would be similar to: 26402 log entries over an elapsed time period of 4936 seconds with a peak of 152 records per second and an overall average of 5 records per second processed 26402 log entries in 77 seconds at a rate of 342 records per second It can be invoked with simply: # lograte < /var/log/syslog Where you can replace /var/log/syslog with whichever file your Lustre logs are going to. If you could run the script (it won't be fast and won't be terribly efficient but I wanted to make it as portable and independent of any scripting tools less common than the bourne shell) on one or more of your logs, paste the output and tell me how many OSSes (in the case of centralized logging) and OSTs each output is for, that would be great. Cheers, and many thanks, b. [1] I do realize that there will be more than just Lustre in those files but it's general logging rates I am looking for, not just the Lustre specific messages -- Brian J. Murrell Senior Software Engineer Whamcloud, Inc.
#!/bin/sh
start_secs=$(date +%s)
n=0
peak_sec_count=0
while read m d t host line; do
log_sec=$(date -d "$m $d $t" +%s)
if [ $n = 0 ]; then
start_sec=$log_sec
this_sec=$log_sec
fi
if [ $this_sec = $log_sec ]; then
this_sec_count=$((this_sec_count+1))
else
this_sec_count=0
fi
if [ $this_sec_count -gt $peak_sec_count ]; then
peak_sec_count=$this_sec_count
fi
elaps_sec=$((log_sec-start_sec))
if [ $elaps_sec -gt 0 ]; then
avg=$(($n/(log_sec-start_sec)))
fi
this_sec=$log_sec
n=$((n+1))
#echo $log_sec $avg $this_sec_count $line
done
spent_secs=$(($(date +%s)-start_secs))
echo "$n log entries over an elapsed time period of $elaps_sec seconds with a
peak of $peak_sec_count records per second and an overall average of $avg
records per second"
echo "processed $n log entries in $spent_secs seconds at a rate of
$((n/spent_secs)) records per second"
signature.asc
Description: OpenPGP digital signature
_______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
