Pat. A couple things I have found:
- How do I get the Rainfall Total line on the Yearly Rainfall graph? - The Yearly Rainfall graph has been factored. Where monthly scales to 2 inches and the max is about .75 inch, the yearly scales to 0.06 inches and the maximum is maybe 0.002 inch? - The graphs are not consistent. Monthly starts at 31 Jul 0:00 except rain, which starts at Monday, July 30 0:00 (different callout format only for rain). Yearly starts at at Jan 1, except for rain that starts at Dec 31. Also, wind direction is missing a callout. Dave On Wed, Aug 29, 2018 at 6:13 AM David Hathaway <[email protected]> wrote: > Hi Gary, > > It doesn't look like the processing time is running away. After starting > at 137 seconds, it eventually went up to 145 seconds before settling back > down to 133 seconds and staying there. This machine has lots of oomph to > give. A Pi might struggle. > > Dave > > On Tue, Aug 28, 2018 at 7:46 PM gjr80 <[email protected]> wrote: > >> Hmm, that is odd. Using WeeWX for 6 odd years on a number of machines I >> have always noticed a drop in times. I guess there is an exception to the >> rule. I guess I need to refactor the SLEs from my highcharts extension but >> that will not be a quick process. If Pat decides to adopt the revised code >> he will need to modify his code as he has made a number of changes to mine. >> >> If you are running a 1 minute archive interval (I think you mentioned >> that earlier?) and have those sorts of execution time you will eventually >> grind to a halt. When a report cycle comes around WeeWX kicks off a new >> thread to run the report engine, if that takes longer than the archive >> interval that thread is still running when the next report thread is >> launched and the extra load slows things down. The process repeats and >> eventually it will grind to a halt with too many report threads open. >> >> Gary >> >> On Wednesday, 29 August 2018 10:37:18 UTC+10, David Hathaway wrote: >>> >>> Gary, >>> I appreciate your response, although much of it is lost on me. However, >>> the idea of a dramatic difference between first pass and subsequent passes >>> isn't happening here. You can tell by the pid that there was a restart, >>> but the times are about the same. >>> >>> Aug 28 17:58:35 hathaway weewx[1319]: cheetahgenerator: Generated 16 >>> files for report Belchertown in 137.68 seconds >>> Aug 28 18:01:35 hathaway weewx[1319]: cheetahgenerator: Generated 16 >>> files for report Belchertown in 137.63 seconds >>> Aug 28 18:04:38 hathaway weewx[1319]: cheetahgenerator: Generated 16 >>> files for report Belchertown in 137.25 seconds >>> Aug 28 18:08:35 hathaway weewx[25730]: cheetahgenerator: Generated 16 >>> files for report Belchertown in 136.85 seconds >>> Aug 28 18:11:34 hathaway weewx[25730]: cheetahgenerator: Generated 16 >>> files for report Belchertown in 137.22 seconds >>> Aug 28 18:14:35 hathaway weewx[25730]: cheetahgenerator: Generated 16 >>> files for report Belchertown in 138.26 seconds >>> >>> Dave >>> >>> >>> >>> On Tue, Aug 28, 2018 at 7:05 PM gjr80 <[email protected]> wrote: >>> >>>> On Tuesday, 28 August 2018 04:11:36 UTC+10, Pat wrote: >>>>> >>>>> Most of the highcharts code is from @gjr80 so he may be able to speak >>>>> better to it, but looking at how it gets its data, it's using >>>>> getSqlVectors >>>>> which looks like it gets its data from the archive table and not a day >>>>> summary table. >>>>> >>>>> >>>> I realise this is a bit dated in terms of this thread but I missed this >>>> post. >>>> >>>> The original 'highcharts' extension I shared with Pat used >>>> getSQLVectors throughout which queries the archive and not the daily >>>> summaries. From memory (and my current highcharts extension has morphed >>>> substantially from what I first shared with Pat) the day and week SLEs >>>> pulled data directly from the archive, month and year used an aggregate >>>> (hour and day respectively I think). This was done for much the same reason >>>> as the default WeeWX plots plot an aggregate for month and year plots - >>>> there is just too many data points otherwise. >>>> >>>> Also, on the topic of execution times. My original highcharts extension >>>> used 4 SLEs to produce 4 json format data files using 1 skin with 4 WeeWX >>>> reports; one report each for day, week, month and year. Each of the SLE >>>> made a number of calls to getSQLVectors to gather the data vectors for >>>> the period concerned, These were not done lazily rather they were executed >>>> in full each time the SLE was called. The way that SLEs work within WeeWX >>>> is that all SLEs listed in skin.conf are executed for each report in >>>> the skin. So even with my 4 reports and 4 SLEs there was quite a lot of >>>> unnecessary SLE execution (16 SLE calls in all when only 4 were actually >>>> required) going on each with numerous complex SQL queries of the archive. >>>> If you now magnify that with Pat's extension where there are quite a few >>>> more reports in the skin that use the highcharts SLEs, the underlying >>>> queries in the SLEs are being run many times each report cycle. When >>>> coupled with a database that has a very short archive interval (say sub-5 >>>> minutes) you magnify the time taken to execute these queries due to the >>>> extra data points in a given period. >>>> >>>> I guess the bottom line is that the highcharts SLEs do not scale well >>>> in a WeeWX report context, judicious placements of reports/SLEs in skins >>>> can help. The highcharts SLEs really need to rewritten to be more efficient >>>> (eg lazy evaluation) but that is a biggish task and has follow on >>>> consequences for the supported reports. >>>> >>>> One quick comment on report times, take note of the time taken for the >>>> second and subsequent report passes after a restart. If the total time >>>> taken for the reports is approaching your archive interval you will >>>> possibly have stability issues. The first report pass after a restart >>>> always takes longer, in my experience up to twice as long as second and >>>> subsequent passes. >>>> >>>> Gary >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "weewx-user" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/weewx-user/GSrKZEiPqsc/unsubscribe. >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "weewx-user" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/weewx-user/GSrKZEiPqsc/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
