Tom, YES, it made a huge difference! With the new xtypes.py all reports now runned in only 102 seconds! Generated 76 images for report vproReport in 101.51 seconds The report was generated with the original skin.conf file with aggregate_type set to max and the aggregate_intervals set.
Thanks! Luc On Saturday, 4 April 2020 09:41:59 UTC-3, Tom Keffer wrote: > > Luc, could you try the attached version of xtypes.py and see if it makes > any difference? > > -tk > > On Sat, Apr 4, 2020 at 5:10 AM Lucas Heijst <[email protected] > <javascript:>> wrote: > >> Tom, Vince, Glenn, >> >> Glenn, >> Yes, I use external mariadb5 databases. >> And no, during the creation of the vector plots the memory use of weewx >> is not much (2.7 %). >> Changing the aggregate_interval has effect, but is not the main cause. >> >> Tom, Vince, Glenn, >> The main cause of the slow vector calculation is the aggregate_type = max >> on windgustvec. >> Without the max aggregation all 5 vector plots (6h, day, week, month, >> year) took together 24 seconds. >> >> Generated 76 images for report vproReport in 113.71 seconds >> >> Luc >> >> ===== >> [[[hourwindvec]]] >> [[[[windvec]]]] >> plot_type = vector >> [[[[windgustvec]]]] >> plot_type = vector >> ### aggregate_type = max >> ### aggregate_interval = 900 # == 15 min >> >> [[[daywindvec]]] >> [[[[windvec]]]] >> plot_type = vector >> [[[[windgustvec]]]] >> plot_type = vector >> ### aggregate_type = max >> ### aggregate_interval = 3600 # == 1 hour >> >> [[[weekwindvec]]] >> [[[[windvec]]]] >> plot_type = vector >> [[[[windgustvec]]]] >> plot_type = vector >> ### aggregate_type = max >> >> [[[monthwindvec]]] >> [[[[windvec]]]] >> plot_type = vector >> [[[[windgustvec]]]] >> plot_type = vector >> ### aggregate_type = max >> ### aggregate_interval = 3600 # == 1 hour >> >> [[[yearwindvec]]] >> [[[[windvec]]]] >> plot_type = vector >> [[[[windgustvec]]]] >> plot_type = vector >> ### aggregate_type = max >> ===== >> >> >> >> >> On Friday, 3 April 2020 22:33:50 UTC-3, Glenn McKechnie wrote: >>> >>> (Sigh, and to the list) >>> >>> Luc, >>> >>> I notice a "Launch of report thread aborted: existing report thread >>> still running" >>> >>> Do you use mysql (mariadb)? >>> If you do, does its CPU usage (from top) increase? >>> >>> If so try dropping the aggregate interval = 900 and see if there is a >>> difference >>> >>> On 04/04/2020, Lucas Heijst <[email protected]> wrote: >>> > Tom, >>> > >>> > There was not much info in the syslog, thats why I didnt send it. >>> > I included the syslog this time. >>> > >>> > I was wrong: the reportgenerator didn't hang, only it is VERY slow >>> > >>> > First I generated all other plots: 71 images in 90 seconds >>> > Generated 71 images for report vproReport in 89.58 seconds >>> > >>> > Then I added a 6-hour vector plot. Extra time for that 6h plot: 70 >>> seconds >>> > Generated 72 images for report vproReport in 159.32 seconds >>> > >>> > Then added a 24-hour vector plot. Extra time for that 24h vector plot >>> 266 >>> > seconds >>> > Generated 73 images for report vproReport in 425.79 seconds >>> > >>> > Estimated time for the other vector plots: >>> > week vector plot 31 minutes >>> > month vector plot: 137 minutes >>> > year vector plot: 27 hours >>> > >>> > Luc >>> > >>> > >>> > On Friday, 3 April 2020 20:13:16 UTC-3, Tom Keffer wrote: >>> >> >>> >> Worked fine for me. >>> >> >>> >> Luc: you know better. We need a log! Perhaps the ReportGenerator is >>> not >>> >> finishing before the next report is due? Perhaps the program >>> segfaulted? >>> >> Who knows without a log? >>> >> >>> >> -tk >>> >> [image: image.png] >>> >> >>> >> On Fri, Apr 3, 2020 at 4:09 PM Lucas Heijst <[email protected] >>> >> <javascript:>> wrote: >>> >> >>> >>> Currently running weewx 4.0.0b18. >>> >>> >>> >>> The reportgenerator hangs (it never finishes) during calculating of >>> the >>> >>> section below. >>> >>> >>> >>> [[[hourwindvec]]] >>> >>> [[[[windvec]]]] >>> >>> plot_type = vector >>> >>> [[[[windgustvec]]]] >>> >>> plot_type = vector >>> >>> aggregate_type = max >>> >>> aggregate_interval = 900 # == 15 min >>> >>> >>> >>> Luc >>> >>> >>> >>> -- >>> >>> You received this message because you are subscribed to the Google >>> Groups >>> >>> >>> >>> "weewx-development" group. >>> >>> To unsubscribe from this group and stop receiving emails from it, >>> send an >>> >>> >>> >>> email to [email protected] <javascript:>. >>> >>> To view this discussion on the web visit >>> >>> >>> https://groups.google.com/d/msgid/weewx-development/601f87df-cbe7-482c-bf49-722a7a5de7bd%40googlegroups.com >>> >>> >>> >>> >>> < >>> https://groups.google.com/d/msgid/weewx-development/601f87df-cbe7-482c-bf49-722a7a5de7bd%40googlegroups.com?utm_medium=email&utm_source=footer> >>> >>> >>> >>> . >>> >>> >>> >> >>> > >>> > -- >>> > You received this message because you are subscribed to the Google >>> Groups >>> > "weewx-development" group. >>> > To unsubscribe from this group and stop receiving emails from it, send >>> an >>> > email to [email protected]. >>> > To view this discussion on the web visit >>> > >>> https://groups.google.com/d/msgid/weewx-development/afee474f-61cb-4c8d-8bfd-5f1c9a18773f%40googlegroups.com. >>> >>> >>> > >>> >>> >>> -- >>> >>> >>> Cheers >>> Glenn >>> >>> rorpi - read only raspberry pi & various weewx addons >>> https://github.com/glennmckechnie >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-development" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/weewx-development/a652d38c-0ed1-4171-9510-5ac601336362%40googlegroups.com >> >> <https://groups.google.com/d/msgid/weewx-development/a652d38c-0ed1-4171-9510-5ac601336362%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "weewx-development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/29a764c9-05ed-4ef1-83c9-9c8e5c603abb%40googlegroups.com.
