I have about 5000 rrd files that I update every 10 minutes. I don't really have
a problem updating the files in that amount of time, but I have to make 12
graphs/file - one set for each of 3 variables showing the data over the last
day, week, month, and year. This obviously is an extremely time and memory
consuming process and I was wondering if there was any way to cut down on the
amount of time needed for this.
Basic algorithm for the script [it's in perl] is:
1) Open the CSV file where I get my data from to figure out the name of
each file
2) Open a file that just contains a number in it that I use to track how
many times the script has been run
3) Update all the last-24-hour graphs because they should be updated every
time new data is collected (10 min)
4) Every third script (ideally 30 min) update the weekly graphs, every
12th script [2 hrs] update the monthly graphs, and once a day [every 72nd
script] update the yearly graphs
5) Increment the number in the file
Obviously, the script does not run anywhere near 10 minutes. I'm open to all
suggestions - I'm not sure if maybe the font change, or the interlaced option,
or anything else makes the graphs take longer to generate. At this point I
don't care much about the quality.
The code:
[code]
print time()."\n";
$rrd = 'C:\RRDTool\RRDTool\Release\rrdtool.exe';
open(INFO, "<Y:\obscurrent.csv") || die('cannot open file: ' . $!);
@lines = <INFO>;
close(INFO);
open(INFO, "<numUpdated.txt") || die('cannot open file: ' . $!);
@lines2 = <INFO>;
close(INFO);
$updatedLast = $lines2[0];
foreach $station (@lines){
@updated = split(/,/,$station);
$cmd= $rrd.' graph images/'.$updated[0].'tempLAST24.png --interlaced -n
DEFAULT:0:C:/WINDOWS/Fonts/ARIAL.TTF -a PNG -X 0 -v Degrees
DEF:theLine=rrd\\'.$updated[0].'.rrd:temp:AVERAGE
LINE1:theLine#FF0000:"Temperature - Last 24 Hours" --start '.(time() - 86400).'
--end N -A';
`$cmd`;
$cmd= $rrd.' graph images/'.$updated[0].'windLAST24.png --interlaced -n
DEFAULT:0:C:/WINDOWS/Fonts/ARIAL.TTF -a PNG -X 0 -v MPH
DEF:theLine=rrd\\'.$updated[0].'.rrd:wind:AVERAGE
LINE1:theLine#FF0000:"Windspeed - Last 24 Hours" --start '.(time() - 86400).'
--end N -A';
`$cmd`;
$cmd= $rrd.' graph images/'.$updated[0].'humidLAST24.png --interlaced -n
DEFAULT:0:C:/WINDOWS/Fonts/ARIAL.TTF -a PNG -X 0 -v Percent
DEF:theLine=rrd\\'.$updated[0].'.rrd:humid:AVERAGE
LINE1:theLine#FF0000:"Humidity - Last 24 Hours" --start '.(time() - 86400).'
--end N -A';
`$cmd`;
#*******************************************************************************
*************************
# UPDATE WEEKLY EVERY 3 TIMES
#*******************************************************************************
*************************
if ($updatedLast% 3 == 0){
$cmd= $rrd.' graph images/'.$updated[0].'tempLASTWEEK.png
--interlaced -n DEFAULT:0:C:/WINDOWS/Fonts/ARIAL.TTF -a PNG -X 0 -v Degrees
DEF:theLine=rrd\\'.$updated[0].'.rrd:temp:AVERAGE
LINE1:theLine#FF0000:"Temperature - Last Week" --start '.(time() - 604800).'
--end N -A';
`$cmd`;
$cmd= $rrd.' graph images/'.$updated[0].'windLASTWEEK.png
--interlaced -n DEFAULT:0:C:/WINDOWS/Fonts/ARIAL.TTF -a PNG -X 0 -v MPH
DEF:theLine=rrd\\'.$updated[0].'.rrd:wind:AVERAGE
LINE1:theLine#FF0000:"Windspeed - Last Week" --start '.(time() - 604800).' --end
N -A';
`$cmd`;
$cmd= $rrd.' graph images/'.$updated[0].'humidLASTWEEK.png
--interlaced -n DEFAULT:0:C:/WINDOWS/Fonts/ARIAL.TTF -a PNG -X 0 -v Percent
DEF:theLine=rrd\\'.$updated[0].'.rrd:humid:AVERAGE
LINE1:theLine#FF0000:"Humidity - Last Week" --start '.(time() - 604800).' --end
N -A';
`$cmd`;
}
#*******************************************************************************
*************************
# UPDATE MONTHLY EVERY 12 TIMES
#*******************************************************************************
*************************
if ($updatedLast% 12 == 0){
$cmd= $rrd.' graph images/'.$updated[0].'tempLASTmonth.png
--interlaced -n DEFAULT:0:C:/WINDOWS/Fonts/ARIAL.TTF -a PNG -X 0 -v Degrees
DEF:theLine=rrd\\'.$updated[0].'.rrd:temp:AVERAGE
LINE1:theLine#FF0000:"Temperature - Last Month" --start '.(time() - 2419200).'
--end N -A';
`$cmd`;
$cmd= $rrd.' graph images/'.$updated[0].'windLASTmonth.png
--interlaced -n DEFAULT:0:C:/WINDOWS/Fonts/ARIAL.TTF -a PNG -X 0 -v MPH
DEF:theLine=rrd\\'.$updated[0].'.rrd:wind:AVERAGE
LINE1:theLine#FF0000:"Windspeed - Last Month" --start '.(time() - 2419200).'
--end N -A';
`$cmd`;
$cmd= $rrd.' graph images/'.$updated[0].'humidLASTmonth.png
--interlaced -n DEFAULT:0:C:/WINDOWS/Fonts/ARIAL.TTF -a PNG -X 0 -v Percent
DEF:theLine=rrd\\'.$updated[0].'.rrd:humid:AVERAGE
LINE1:theLine#FF0000:"Humidity - Last Month" --start '.(time() - 2419200).'
--end N -A';
`$cmd`;
}
#*******************************************************************************
*************************
# UPDATE YEARLY EVERY 72 TIMES
#*******************************************************************************
*************************
if ($updatedLast% 72 == 0){
$cmd= $rrd.' graph images/'.$updated[0].'tempLASTyear.png
--interlaced -n DEFAULT:0:C:/WINDOWS/Fonts/ARIAL.TTF -a PNG -X 0 -v Degrees
DEF:theLine=rrd\\'.$updated[0].'.rrd:temp:AVERAGE
LINE1:theLine#FF0000:"Temperature - Last Year" --start '.(time() - 31536000).'
--end N -A';
`$cmd`;
$cmd= $rrd.' graph images/'.$updated[0].'windLASTyear.png
--interlaced -n DEFAULT:0:C:/WINDOWS/Fonts/ARIAL.TTF -a PNG -X 0 -v MPH
DEF:theLine=rrd\\'.$updated[0].'.rrd:wind:AVERAGE
LINE1:theLine#FF0000:"Windspeed - Last Year" --start '.(time() - 31536000).'
--end N -A';
`$cmd`;
$cmd= $rrd.' graph images/'.$updated[0].'humidLASTyear.png
--interlaced -n DEFAULT:0:C:/WINDOWS/Fonts/ARIAL.TTF -a PNG -X 0 -v Percent
DEF:theLine=rrd\\'.$updated[0].'.rrd:humid:AVERAGE
LINE1:theLine#FF0000:"Humidity - Last Year" --start '.(time() - 31536000).'
--end N -A';
`$cmd`;
}
}
$file = 'numUpdated.txt';
open(INFO, ">$file");
print INFO ($updated+1);
close(INFO);
print time();
[/code]
This message (including any attachments) is a confidential and privileged
communication of
AWS Convergence Technologies, Inc. and intended only for the addressee. Any
unauthorized use,
distribution or copying of this message (or any attachment) is prohibited. If
you are not the
addressee or a person authorized to receive messages for the addressee, you
have received this
message in error. In that case, please delete this message and call us at
301-250-4000 so that we
can correct our records in order to avoid this mistake in the future. Thank you.
--
Unsubscribe mailto:[EMAIL PROTECTED]
Help mailto:[EMAIL PROTECTED]
Archive http://lists.ee.ethz.ch/rrd-users
WebAdmin http://lists.ee.ethz.ch/lsg2.cgi
