On Tuesday, 28 August 2018 04:11:36 UTC+10, Pat wrote: > > Most of the highcharts code is from @gjr80 so he may be able to speak > better to it, but looking at how it gets its data, it's using getSqlVectors > which looks like it gets its data from the archive table and not a day > summary table. > > I realise this is a bit dated in terms of this thread but I missed this post.
The original 'highcharts' extension I shared with Pat used getSQLVectors throughout which queries the archive and not the daily summaries. From memory (and my current highcharts extension has morphed substantially from what I first shared with Pat) the day and week SLEs pulled data directly from the archive, month and year used an aggregate (hour and day respectively I think). This was done for much the same reason as the default WeeWX plots plot an aggregate for month and year plots - there is just too many data points otherwise. Also, on the topic of execution times. My original highcharts extension used 4 SLEs to produce 4 json format data files using 1 skin with 4 WeeWX reports; one report each for day, week, month and year. Each of the SLE made a number of calls to getSQLVectors to gather the data vectors for the period concerned, These were not done lazily rather they were executed in full each time the SLE was called. The way that SLEs work within WeeWX is that all SLEs listed in skin.conf are executed for each report in the skin. So even with my 4 reports and 4 SLEs there was quite a lot of unnecessary SLE execution (16 SLE calls in all when only 4 were actually required) going on each with numerous complex SQL queries of the archive. If you now magnify that with Pat's extension where there are quite a few more reports in the skin that use the highcharts SLEs, the underlying queries in the SLEs are being run many times each report cycle. When coupled with a database that has a very short archive interval (say sub-5 minutes) you magnify the time taken to execute these queries due to the extra data points in a given period. I guess the bottom line is that the highcharts SLEs do not scale well in a WeeWX report context, judicious placements of reports/SLEs in skins can help. The highcharts SLEs really need to rewritten to be more efficient (eg lazy evaluation) but that is a biggish task and has follow on consequences for the supported reports. One quick comment on report times, take note of the time taken for the second and subsequent report passes after a restart. If the total time taken for the reports is approaching your archive interval you will possibly have stability issues. The first report pass after a restart always takes longer, in my experience up to twice as long as second and subsequent passes. Gary -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
