Re: [weewx-development] Reportgenerator 'hangs' with vector plot

2020-04-04 Thread Greg Troxel
mwall  writes:

> we started a postgres adapter years ago.  it never made it into weewx core.

Sorry -- I use a lot of programs and my memory was that weewx supported
postgres.  I have used it for various things over the years including
one project that had frequent crashes due to laptop batteries dying and
it did amazingly well in terms of robustness.

> the weewx column names are mixed case, like outTemp and windSpeed.  as i 
> recall, postgres lower-cases everything that is not quoted.  so we would 
> need a general way of dealing with case sensitivity that works with mysql, 
> sqlite, and postgres.

Interesting - I am guessing that postgres is compliant with the SQL
specs, but making a different within-spec choice.

> i'm pretty sure it would not be too difficult for weewx core, but making it 
> general enough to work for search list extensions and cheetah templates 
> could be a problem (i think that is why it never made it into weewx core).


> if you want to hack on it, make a postgresql adapter in bin/weedb/pgsql.py 
> - follow the pattern in sqlite.py or mysql.py in that directory.

I have put your text into my todo list.   But it's large :-(

Thanks for taking the time to straighten me out.

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-development+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-development/rmio8s7atmi.fsf%40s1.lexort.com.


Re: [weewx-development] Reportgenerator 'hangs' with vector plot

2020-04-04 Thread mwall
On Saturday, April 4, 2020 at 9:24:41 AM UTC-4, Greg Troxel wrote:
>
>
> I wonder if people have tried postgresql.  (I'm still on sqlite.) 
>

here is a summary of how identifiers are treated in various databases:

https://github.com/ontop/ontop/wiki/Case-sensitivity-for-SQL-identifiers 

the original discussion in weewx user group:

https://groups.google.com/forum/#!msg/weewx-development/5fzYoRio2aM/P9qgDqHBEjgJ

a related issue that still exists in mysql (when running on windows or 
macos):

https://github.com/weewx/weewx/issues/27


-- 
You received this message because you are subscribed to the Google Groups 
"weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-development+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-development/549f14a1-2982-41dd-a68b-d8e0924eeb05%40googlegroups.com.


Re: [weewx-development] Reportgenerator 'hangs' with vector plot

2020-04-04 Thread mwall
On Saturday, April 4, 2020 at 9:24:41 AM UTC-4, Greg Troxel wrote:
>
> Glenn McKechnie > writes: 
>
> > It's not weewx increasing in CPU usage, it's the CPU usage for 
> > mariadb. Check on your Synology NAS. 
>
> I wonder if people have tried postgresql.  (I'm still on sqlite.) 
>

greg, 

we started a postgres adapter years ago.  it never made it into weewx core.

the weewx column names are mixed case, like outTemp and windSpeed.  as i 
recall, postgres lower-cases everything that is not quoted.  so we would 
need a general way of dealing with case sensitivity that works with mysql, 
sqlite, and postgres.

i'm pretty sure it would not be too difficult for weewx core, but making it 
general enough to work for search list extensions and cheetah templates 
could be a problem (i think that is why it never made it into weewx core).

if you want to hack on it, make a postgresql adapter in bin/weedb/pgsql.py 
- follow the pattern in sqlite.py or mysql.py in that directory.

m

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-development+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-development/3239f908-b392-4855-807d-36d1da319b8f%40googlegroups.com.


Re: [weewx-development] Reportgenerator 'hangs' with vector plot

2020-04-04 Thread Lucas Heijst
Tom,

The plots look good to me, see attachments.

Luc

On Saturday, 4 April 2020 10:19:06 UTC-3, Tom Keffer wrote:
>
> Before you get too excited, make sure the fix still gives the right 
> answer. It uses a different query strategy.
>
> On Sat, Apr 4, 2020 at 6:17 AM Glenn McKechnie  > wrote:
>
>> Well, that fixed it on this machine!
>>
>> Back to the 16% mysqld usage with both 'aggregate_type = max' and
>> 'aggregate_interval = 300' reinstated in skin.conf
>>
>> Wouldn't have a fix as smooth as that for covid-19 would you?
>>
>>
>>
>> On 04/04/2020, Thomas Keffer > wrote:
>> > Luc, could you try the attached version of xtypes.py and see if it makes
>> > any difference?
>> >
>> > -tk
>> >
>> > On Sat, Apr 4, 2020 at 5:10 AM Lucas Heijst > > wrote:
>> >
>> >> Tom, Vince, Glenn,
>> >>
>> >> Glenn,
>> >> Yes, I use external mariadb5 databases.
>> >> And no, during the creation of the vector plots the memory use of weewx
>> >> is
>> >> not much (2.7 %).
>> >> Changing the aggregate_interval has effect, but is not the main cause.
>> >>
>> >> Tom, Vince, Glenn,
>> >> The main cause of the slow vector calculation is the aggregate_type = 
>> max
>> >> on windgustvec.
>> >> Without the max aggregation all 5 vector plots (6h, day, week, month,
>> >> year) took together 24 seconds.
>> >>
>> >> Generated 76 images for report vproReport in 113.71 seconds
>> >>
>> >> Luc
>> >>
>> >> =
>> >> [[[hourwindvec]]]
>> >> windvec
>> >> plot_type  = vector
>> >> windgustvec
>> >> plot_type  = vector
>> >> ###aggregate_type = max
>> >> ###aggregate_interval = 900# == 15 min
>> >>
>> >> [[[daywindvec]]]
>> >> windvec
>> >> plot_type= vector
>> >> windgustvec
>> >> plot_type= vector
>> >> ###aggregate_type   = max
>> >> ###aggregate_interval   = 3600# == 1 hour
>> >>
>> >> [[[weekwindvec]]]
>> >> windvec
>> >> plot_type= vector
>> >> windgustvec
>> >> plot_type= vector
>> >> ###aggregate_type   = max
>> >>
>> >> [[[monthwindvec]]]
>> >> windvec
>> >> plot_type= vector
>> >> windgustvec
>> >> plot_type= vector
>> >> ###aggregate_type   = max
>> >> ###aggregate_interval   = 3600# == 1 hour
>> >>
>> >> [[[yearwindvec]]]
>> >> windvec
>> >> plot_type= vector
>> >> windgustvec
>> >> plot_type= vector
>> >> ###aggregate_type   = max
>> >> =
>> >>
>> >>
>> >>
>> >>
>> >> On Friday, 3 April 2020 22:33:50 UTC-3, Glenn McKechnie wrote:
>> >>>
>> >>> (Sigh, and to the list)
>> >>>
>> >>> Luc,
>> >>>
>> >>> I notice a "Launch of report thread aborted: existing report thread
>> >>> still running"
>> >>>
>> >>> Do you use mysql (mariadb)?
>> >>> If you do, does its CPU usage (from top) increase?
>> >>>
>> >>> If so try dropping the aggregate interval = 900 and see if there is a
>> >>> difference
>> >>>
>> >>> On 04/04/2020, Lucas Heijst  wrote:
>> >>> > Tom,
>> >>> >
>> >>> > There was not much info in the syslog, thats why I didnt send it.
>> >>> > I included the syslog this time.
>> >>> >
>> >>> > I was wrong: the reportgenerator didn't hang, only it is VERY slow
>> >>> >
>> >>> > First I generated all other plots: 71 images in 90 seconds
>> >>> > Generated 71 images for report vproReport in 89.58 seconds
>> >>> >
>> >>> > Then I added a 6-hour vector plot. Extra time for that 6h plot: 70
>> >>> seconds
>> >>> > Generated 72 images for report vproReport in 159.32 seconds
>> >>> >
>> >>> > Then added a 24-hour vector plot. Extra time for that 24h vector 
>> plot
>> >>> 266
>> >>> > seconds
>> >>> > Generated 73 images for report vproReport in 425.79 seconds
>> >>> >
>> >>> > Estimated time for the other vector plots:
>> >>> > week vector plot 31 minutes
>> >>> > month vector plot: 137 minutes
>> >>> > year vector plot: 27 hours
>> >>> >
>> >>> > Luc
>> >>> >
>> >>> >
>> >>> > On Friday, 3 April 2020 20:13:16 UTC-3, Tom Keffer wrote:
>> >>> >>
>> >>> >> Worked fine for me.
>> >>> >>
>> >>> >> Luc: you know better. We need a log! Perhaps the ReportGenerator is
>> >>> not
>> >>> >> finishing before the next report is due? Perhaps the program
>> >>> segfaulted?
>> >>> >> Who knows without a log?
>> >>> >>
>> >>> >> -tk
>> >>> >> [image: image.png]
>> >>> >>
>> >>> >> On Fri, Apr 3, 2020 at 4:09 PM Lucas Heijst > >>> >> > wrote:
>> >>> >>
>> >>> >>> Currently running weewx 4.0.0b18.
>> >>> >>>
>> >>> >>> The reportgenerator hangs (it never finishes) during calculating 
>> of
>> 

Re: [weewx-development] Reportgenerator 'hangs' with vector plot

2020-04-04 Thread Lucas Heijst
Glenn,

The change in Toms xtypes.py is the way how the database query is done for 
max and min aggregation
So I assume the old query kept the DB manager very busy (but I did not 
check it because in the new version the query is fast).

Luc

On Saturday, 4 April 2020 10:03:13 UTC-3, Glenn McKechnie wrote:
>
> Luc, 
>
> It's not weewx increasing in CPU usage, it's the CPU usage for 
> mariadb. Check on your Synology NAS. 
>
> Maybe it's not the same for yours, but I found this occurring in mine. 
>
> Worked in 3.9.2, failed in weewx 4.0.06b where it sent the CPU through 
> the roof, with the reports failing in the same manner as yours. 
> I assumed mariadb had become a choke point, commenting it out (as 
> you've done) brought things back to 'normal'. 
>
> In fact, looking through my skin.conf shows the note I've left myself. 
> I see this. 
>
> [[[daywindvec]]] 
>#-# see note below 
>#-# aggregate_interval = 300 
> windvec 
> label = Wind Vector (5 min avg.) 
> plot_type = vector 
> aggregate_type = avg 
> aggregate_interval = 300 
> #windgustvec 
> #label = Gust Wind 
> #plot_type = vector 
> ###aggregate_type = max 
> #-# mysqld goes through the roof if this is enabled 
> #-# aggregate_interval = 300 
>
> Obviously I haven't got back to it. Life (and fires) have a knack of 
> getting in the way sometimes. 
>
> Okay, Quick test here: 
> CPU for mysqld with no report thread running  0.2% 
> CPU for mysqld with the (seasons) skin.conf as above 16.2% 
> Now... 
> Uncommenting those entries (windgustvec) sends CPU to 163% (114%, 133% 
> 155% etc) mysqld is seriously struggling with it. I have to revert 
> those changes and restart mysqld to get it back down again. 
>
> Taking it a little further it's the, not the 'aggregate_interval = 
> 300' that sets it off it's the 'aggregate_type = max ' 
>
> Changing that to  'aggregate_type =  avg' and it's okay, commenting it 
> out completely is also okay. 
>
> Leaving it uncommented (activated) is poison. 
> Wait for an archive cycle in this high usage state and the CPU is 
> still maxed out and my wee_reports fail to cycle. That means that with 
> my quirky setup it's still running and hasn't finished or been reaped. 
> Interestingly, when I restart weewx that separate wee_report instance 
> completes. 
>
>
> On 04/04/2020, Lucas Heijst > wrote: 
> > Tom, Vince, Glenn, 
> > 
> > Glenn, 
> > Yes, I use external mariadb5 databases. 
> > And no, during the creation of the vector plots the memory use of weewx 
> is 
> > not much (2.7 %). 
> > Changing the aggregate_interval has effect, but is not the main cause. 
> > 
> > Tom, Vince, Glenn, 
> > The main cause of the slow vector calculation is the aggregate_type = 
> max 
> > on windgustvec. 
> > Without the max aggregation all 5 vector plots (6h, day, week, month, 
> year) 
> > 
> > took together 24 seconds. 
> > 
> > Generated 76 images for report vproReport in 113.71 seconds 
> > 
> > Luc 
> > 
> > = 
> > [[[hourwindvec]]] 
> > windvec 
> > plot_type  = vector 
> > windgustvec 
> > plot_type  = vector 
> > ###aggregate_type = max 
> > ###aggregate_interval = 900# == 15 min 
> > 
> > [[[daywindvec]]] 
> > windvec 
> > plot_type= vector 
> > windgustvec 
> > plot_type= vector 
> > ###aggregate_type   = max 
> > ###aggregate_interval   = 3600# == 1 hour 
> > 
> > [[[weekwindvec]]] 
> > windvec 
> > plot_type= vector 
> > windgustvec 
> > plot_type= vector 
> > ###aggregate_type   = max 
> > 
> > [[[monthwindvec]]] 
> > windvec 
> > plot_type= vector 
> > windgustvec 
> > plot_type= vector 
> > ###aggregate_type   = max 
> > ###aggregate_interval   = 3600# == 1 hour 
> > 
> > [[[yearwindvec]]] 
> > windvec 
> > plot_type= vector 
> > windgustvec 
> > plot_type= vector 
> > ###aggregate_type   = max 
> > = 
> > 
> > 
> > 
> > 
> > On Friday, 3 April 2020 22:33:50 UTC-3, Glenn McKechnie wrote: 
> >> 
> >> (Sigh, and to the list) 
> >> 
> >> Luc, 
> >> 
> >> I notice a "Launch of report thread aborted: existing report thread 
> >> still running" 
> >> 
> >> Do you use mysql (mariadb)? 
> >> If you do, does its CPU usage (from top) increase? 
> 

Re: [weewx-development] Reportgenerator 'hangs' with vector plot

2020-04-04 Thread Glenn McKechnie
On 05/04/2020, Thomas Keffer  wrote:
> Before you get too excited, make sure the fix still gives the right answer.
> It uses a different query strategy.

My first thought was how can I tell without firing up a 3.9.2 version
(Although I think I might have access to one of those :-)

But then if I change the values on this one...
Attached are 3 charts - all at max,  None aggregate interval, 300
aggregate interval and 900 aggregate interval. Seem okay - ish


-- 

Cheers
 Glenn

rorpi - read only raspberry pi & various weewx addons
https://github.com/glennmckechnie

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-development+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-development/CAAraAzgGEZZXmW2TfrBiSMM%2Bidn8mN87h3%3D%2Bcs6o_m6FjF_WHg%40mail.gmail.com.


Re: [weewx-development] Reportgenerator 'hangs' with vector plot

2020-04-04 Thread Lucas Heijst
Tom,

YES, it made a huge difference! 
With the new xtypes.py all reports now runned in only 102 seconds!
Generated 76 images for report vproReport in 101.51 seconds
The report was generated with the original skin.conf file with 
aggregate_type set to max and the aggregate_intervals set.

Thanks!

Luc


On Saturday, 4 April 2020 09:41:59 UTC-3, Tom Keffer wrote:
>
> Luc, could you try the attached version of xtypes.py and see if it makes 
> any difference?
>
> -tk
>
> On Sat, Apr 4, 2020 at 5:10 AM Lucas Heijst  > wrote:
>
>> Tom, Vince, Glenn,
>>
>> Glenn,
>> Yes, I use external mariadb5 databases.
>> And no, during the creation of the vector plots the memory use of weewx 
>> is not much (2.7 %).
>> Changing the aggregate_interval has effect, but is not the main cause.
>>
>> Tom, Vince, Glenn,
>> The main cause of the slow vector calculation is the aggregate_type = max 
>> on windgustvec.
>> Without the max aggregation all 5 vector plots (6h, day, week, month, 
>> year) took together 24 seconds.
>>
>> Generated 76 images for report vproReport in 113.71 seconds
>>
>> Luc
>>
>> =
>> [[[hourwindvec]]]
>> windvec
>> plot_type  = vector
>> windgustvec
>> plot_type  = vector
>> ###aggregate_type = max
>> ###aggregate_interval = 900# == 15 min
>>
>> [[[daywindvec]]]
>> windvec
>> plot_type= vector
>> windgustvec
>> plot_type= vector
>> ###aggregate_type   = max
>> ###aggregate_interval   = 3600# == 1 hour
>>
>> [[[weekwindvec]]]
>> windvec
>> plot_type= vector
>> windgustvec
>> plot_type= vector
>> ###aggregate_type   = max
>>
>> [[[monthwindvec]]]
>> windvec
>> plot_type= vector
>> windgustvec
>> plot_type= vector
>> ###aggregate_type   = max
>> ###aggregate_interval   = 3600# == 1 hour
>>
>> [[[yearwindvec]]]
>> windvec
>> plot_type= vector
>> windgustvec
>> plot_type= vector
>> ###aggregate_type   = max
>> =
>>
>>
>>
>>
>> On Friday, 3 April 2020 22:33:50 UTC-3, Glenn McKechnie wrote:
>>>
>>> (Sigh, and to the list) 
>>>
>>> Luc, 
>>>
>>> I notice a "Launch of report thread aborted: existing report thread 
>>> still running" 
>>>
>>> Do you use mysql (mariadb)? 
>>> If you do, does its CPU usage (from top) increase? 
>>>
>>> If so try dropping the aggregate interval = 900 and see if there is a 
>>> difference 
>>>
>>> On 04/04/2020, Lucas Heijst  wrote: 
>>> > Tom, 
>>> > 
>>> > There was not much info in the syslog, thats why I didnt send it. 
>>> > I included the syslog this time. 
>>> > 
>>> > I was wrong: the reportgenerator didn't hang, only it is VERY slow 
>>> > 
>>> > First I generated all other plots: 71 images in 90 seconds 
>>> > Generated 71 images for report vproReport in 89.58 seconds 
>>> > 
>>> > Then I added a 6-hour vector plot. Extra time for that 6h plot: 70 
>>> seconds 
>>> > Generated 72 images for report vproReport in 159.32 seconds 
>>> > 
>>> > Then added a 24-hour vector plot. Extra time for that 24h vector plot 
>>> 266 
>>> > seconds 
>>> > Generated 73 images for report vproReport in 425.79 seconds 
>>> > 
>>> > Estimated time for the other vector plots: 
>>> > week vector plot 31 minutes 
>>> > month vector plot: 137 minutes 
>>> > year vector plot: 27 hours 
>>> > 
>>> > Luc 
>>> > 
>>> > 
>>> > On Friday, 3 April 2020 20:13:16 UTC-3, Tom Keffer wrote: 
>>> >> 
>>> >> Worked fine for me. 
>>> >> 
>>> >> Luc: you know better. We need a log! Perhaps the ReportGenerator is 
>>> not 
>>> >> finishing before the next report is due? Perhaps the program 
>>> segfaulted? 
>>> >> Who knows without a log? 
>>> >> 
>>> >> -tk 
>>> >> [image: image.png] 
>>> >> 
>>> >> On Fri, Apr 3, 2020 at 4:09 PM Lucas Heijst >> >> > wrote: 
>>> >> 
>>> >>> Currently running weewx 4.0.0b18. 
>>> >>> 
>>> >>> The reportgenerator hangs (it never finishes) during calculating of 
>>> the 
>>> >>> section below. 
>>> >>> 
>>> >>> [[[hourwindvec]]] 
>>> >>> windvec 
>>> >>> plot_type  = vector 
>>> >>> windgustvec 
>>> >>> plot_type  = vector 
>>> >>> aggregate_type = max 
>>> >>> aggregate_interval = 900# == 15 min 
>>> >>> 
>>> >>> Luc 
>>> >>> 
>>> >>> -- 
>>> >>> You received this message because you are subscribed to the Google 
>>> Groups 
>>> >>> 
>>> >>> "weewx-development" group. 
>>> >>> To 

Re: [weewx-development] Reportgenerator 'hangs' with vector plot

2020-04-04 Thread Greg Troxel
Glenn McKechnie  writes:

> It's not weewx increasing in CPU usage, it's the CPU usage for
> mariadb. Check on your Synology NAS.

I wonder if people have tried postgresql.  (I'm still on sqlite.)

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-development+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-development/rmieet3crjc.fsf%40s1.lexort.com.


Re: [weewx-development] Reportgenerator 'hangs' with vector plot

2020-04-04 Thread Thomas Keffer
Before you get too excited, make sure the fix still gives the right answer.
It uses a different query strategy.

On Sat, Apr 4, 2020 at 6:17 AM Glenn McKechnie 
wrote:

> Well, that fixed it on this machine!
>
> Back to the 16% mysqld usage with both 'aggregate_type = max' and
> 'aggregate_interval = 300' reinstated in skin.conf
>
> Wouldn't have a fix as smooth as that for covid-19 would you?
>
>
>
> On 04/04/2020, Thomas Keffer  wrote:
> > Luc, could you try the attached version of xtypes.py and see if it makes
> > any difference?
> >
> > -tk
> >
> > On Sat, Apr 4, 2020 at 5:10 AM Lucas Heijst 
> wrote:
> >
> >> Tom, Vince, Glenn,
> >>
> >> Glenn,
> >> Yes, I use external mariadb5 databases.
> >> And no, during the creation of the vector plots the memory use of weewx
> >> is
> >> not much (2.7 %).
> >> Changing the aggregate_interval has effect, but is not the main cause.
> >>
> >> Tom, Vince, Glenn,
> >> The main cause of the slow vector calculation is the aggregate_type =
> max
> >> on windgustvec.
> >> Without the max aggregation all 5 vector plots (6h, day, week, month,
> >> year) took together 24 seconds.
> >>
> >> Generated 76 images for report vproReport in 113.71 seconds
> >>
> >> Luc
> >>
> >> =
> >> [[[hourwindvec]]]
> >> windvec
> >> plot_type  = vector
> >> windgustvec
> >> plot_type  = vector
> >> ###aggregate_type = max
> >> ###aggregate_interval = 900# == 15 min
> >>
> >> [[[daywindvec]]]
> >> windvec
> >> plot_type= vector
> >> windgustvec
> >> plot_type= vector
> >> ###aggregate_type   = max
> >> ###aggregate_interval   = 3600# == 1 hour
> >>
> >> [[[weekwindvec]]]
> >> windvec
> >> plot_type= vector
> >> windgustvec
> >> plot_type= vector
> >> ###aggregate_type   = max
> >>
> >> [[[monthwindvec]]]
> >> windvec
> >> plot_type= vector
> >> windgustvec
> >> plot_type= vector
> >> ###aggregate_type   = max
> >> ###aggregate_interval   = 3600# == 1 hour
> >>
> >> [[[yearwindvec]]]
> >> windvec
> >> plot_type= vector
> >> windgustvec
> >> plot_type= vector
> >> ###aggregate_type   = max
> >> =
> >>
> >>
> >>
> >>
> >> On Friday, 3 April 2020 22:33:50 UTC-3, Glenn McKechnie wrote:
> >>>
> >>> (Sigh, and to the list)
> >>>
> >>> Luc,
> >>>
> >>> I notice a "Launch of report thread aborted: existing report thread
> >>> still running"
> >>>
> >>> Do you use mysql (mariadb)?
> >>> If you do, does its CPU usage (from top) increase?
> >>>
> >>> If so try dropping the aggregate interval = 900 and see if there is a
> >>> difference
> >>>
> >>> On 04/04/2020, Lucas Heijst  wrote:
> >>> > Tom,
> >>> >
> >>> > There was not much info in the syslog, thats why I didnt send it.
> >>> > I included the syslog this time.
> >>> >
> >>> > I was wrong: the reportgenerator didn't hang, only it is VERY slow
> >>> >
> >>> > First I generated all other plots: 71 images in 90 seconds
> >>> > Generated 71 images for report vproReport in 89.58 seconds
> >>> >
> >>> > Then I added a 6-hour vector plot. Extra time for that 6h plot: 70
> >>> seconds
> >>> > Generated 72 images for report vproReport in 159.32 seconds
> >>> >
> >>> > Then added a 24-hour vector plot. Extra time for that 24h vector plot
> >>> 266
> >>> > seconds
> >>> > Generated 73 images for report vproReport in 425.79 seconds
> >>> >
> >>> > Estimated time for the other vector plots:
> >>> > week vector plot 31 minutes
> >>> > month vector plot: 137 minutes
> >>> > year vector plot: 27 hours
> >>> >
> >>> > Luc
> >>> >
> >>> >
> >>> > On Friday, 3 April 2020 20:13:16 UTC-3, Tom Keffer wrote:
> >>> >>
> >>> >> Worked fine for me.
> >>> >>
> >>> >> Luc: you know better. We need a log! Perhaps the ReportGenerator is
> >>> not
> >>> >> finishing before the next report is due? Perhaps the program
> >>> segfaulted?
> >>> >> Who knows without a log?
> >>> >>
> >>> >> -tk
> >>> >> [image: image.png]
> >>> >>
> >>> >> On Fri, Apr 3, 2020 at 4:09 PM Lucas Heijst  >>> >> > wrote:
> >>> >>
> >>> >>> Currently running weewx 4.0.0b18.
> >>> >>>
> >>> >>> The reportgenerator hangs (it never finishes) during calculating of
> >>> the
> >>> >>> section below.
> >>> >>>
> >>> >>> [[[hourwindvec]]]
> >>> >>> windvec
> >>> >>> plot_type  = vector
> >>> >>> windgustvec
> >>> >>> plot_type  = vector
> >>> >>> 

Re: [weewx-development] Reportgenerator 'hangs' with vector plot

2020-04-04 Thread Glenn McKechnie
Well, that fixed it on this machine!

Back to the 16% mysqld usage with both 'aggregate_type = max' and
'aggregate_interval = 300' reinstated in skin.conf

Wouldn't have a fix as smooth as that for covid-19 would you?



On 04/04/2020, Thomas Keffer  wrote:
> Luc, could you try the attached version of xtypes.py and see if it makes
> any difference?
>
> -tk
>
> On Sat, Apr 4, 2020 at 5:10 AM Lucas Heijst  wrote:
>
>> Tom, Vince, Glenn,
>>
>> Glenn,
>> Yes, I use external mariadb5 databases.
>> And no, during the creation of the vector plots the memory use of weewx
>> is
>> not much (2.7 %).
>> Changing the aggregate_interval has effect, but is not the main cause.
>>
>> Tom, Vince, Glenn,
>> The main cause of the slow vector calculation is the aggregate_type = max
>> on windgustvec.
>> Without the max aggregation all 5 vector plots (6h, day, week, month,
>> year) took together 24 seconds.
>>
>> Generated 76 images for report vproReport in 113.71 seconds
>>
>> Luc
>>
>> =
>> [[[hourwindvec]]]
>> windvec
>> plot_type  = vector
>> windgustvec
>> plot_type  = vector
>> ###aggregate_type = max
>> ###aggregate_interval = 900# == 15 min
>>
>> [[[daywindvec]]]
>> windvec
>> plot_type= vector
>> windgustvec
>> plot_type= vector
>> ###aggregate_type   = max
>> ###aggregate_interval   = 3600# == 1 hour
>>
>> [[[weekwindvec]]]
>> windvec
>> plot_type= vector
>> windgustvec
>> plot_type= vector
>> ###aggregate_type   = max
>>
>> [[[monthwindvec]]]
>> windvec
>> plot_type= vector
>> windgustvec
>> plot_type= vector
>> ###aggregate_type   = max
>> ###aggregate_interval   = 3600# == 1 hour
>>
>> [[[yearwindvec]]]
>> windvec
>> plot_type= vector
>> windgustvec
>> plot_type= vector
>> ###aggregate_type   = max
>> =
>>
>>
>>
>>
>> On Friday, 3 April 2020 22:33:50 UTC-3, Glenn McKechnie wrote:
>>>
>>> (Sigh, and to the list)
>>>
>>> Luc,
>>>
>>> I notice a "Launch of report thread aborted: existing report thread
>>> still running"
>>>
>>> Do you use mysql (mariadb)?
>>> If you do, does its CPU usage (from top) increase?
>>>
>>> If so try dropping the aggregate interval = 900 and see if there is a
>>> difference
>>>
>>> On 04/04/2020, Lucas Heijst  wrote:
>>> > Tom,
>>> >
>>> > There was not much info in the syslog, thats why I didnt send it.
>>> > I included the syslog this time.
>>> >
>>> > I was wrong: the reportgenerator didn't hang, only it is VERY slow
>>> >
>>> > First I generated all other plots: 71 images in 90 seconds
>>> > Generated 71 images for report vproReport in 89.58 seconds
>>> >
>>> > Then I added a 6-hour vector plot. Extra time for that 6h plot: 70
>>> seconds
>>> > Generated 72 images for report vproReport in 159.32 seconds
>>> >
>>> > Then added a 24-hour vector plot. Extra time for that 24h vector plot
>>> 266
>>> > seconds
>>> > Generated 73 images for report vproReport in 425.79 seconds
>>> >
>>> > Estimated time for the other vector plots:
>>> > week vector plot 31 minutes
>>> > month vector plot: 137 minutes
>>> > year vector plot: 27 hours
>>> >
>>> > Luc
>>> >
>>> >
>>> > On Friday, 3 April 2020 20:13:16 UTC-3, Tom Keffer wrote:
>>> >>
>>> >> Worked fine for me.
>>> >>
>>> >> Luc: you know better. We need a log! Perhaps the ReportGenerator is
>>> not
>>> >> finishing before the next report is due? Perhaps the program
>>> segfaulted?
>>> >> Who knows without a log?
>>> >>
>>> >> -tk
>>> >> [image: image.png]
>>> >>
>>> >> On Fri, Apr 3, 2020 at 4:09 PM Lucas Heijst >> >> > wrote:
>>> >>
>>> >>> Currently running weewx 4.0.0b18.
>>> >>>
>>> >>> The reportgenerator hangs (it never finishes) during calculating of
>>> the
>>> >>> section below.
>>> >>>
>>> >>> [[[hourwindvec]]]
>>> >>> windvec
>>> >>> plot_type  = vector
>>> >>> windgustvec
>>> >>> plot_type  = vector
>>> >>> aggregate_type = max
>>> >>> aggregate_interval = 900# == 15 min
>>> >>>
>>> >>> Luc
>>> >>>
>>> >>> --
>>> >>> You received this message because you are subscribed to the Google
>>> Groups
>>> >>>
>>> >>> "weewx-development" group.
>>> >>> To unsubscribe from this group and stop receiving emails from it,
>>> send an
>>> >>>
>>> >>> email to weewx-de...@googlegroups.com .
>>> >>> To view this discussion on the web visit
>>> >>>
>>> 

Re: [weewx-development] Reportgenerator 'hangs' with vector plot

2020-04-04 Thread Glenn McKechnie
Luc,

It's not weewx increasing in CPU usage, it's the CPU usage for
mariadb. Check on your Synology NAS.

Maybe it's not the same for yours, but I found this occurring in mine.

Worked in 3.9.2, failed in weewx 4.0.06b where it sent the CPU through
the roof, with the reports failing in the same manner as yours.
I assumed mariadb had become a choke point, commenting it out (as
you've done) brought things back to 'normal'.

In fact, looking through my skin.conf shows the note I've left myself.
I see this.

[[[daywindvec]]]
   #-# see note below
   #-# aggregate_interval = 300
windvec
label = Wind Vector (5 min avg.)
plot_type = vector
aggregate_type = avg
aggregate_interval = 300
#windgustvec
#label = Gust Wind
#plot_type = vector
###aggregate_type = max
#-# mysqld goes through the roof if this is enabled
#-# aggregate_interval = 300

Obviously I haven't got back to it. Life (and fires) have a knack of
getting in the way sometimes.

Okay, Quick test here:
CPU for mysqld with no report thread running  0.2%
CPU for mysqld with the (seasons) skin.conf as above 16.2%
Now...
Uncommenting those entries (windgustvec) sends CPU to 163% (114%, 133%
155% etc) mysqld is seriously struggling with it. I have to revert
those changes and restart mysqld to get it back down again.

Taking it a little further it's the, not the 'aggregate_interval =
300' that sets it off it's the 'aggregate_type = max '

Changing that to  'aggregate_type =  avg' and it's okay, commenting it
out completely is also okay.

Leaving it uncommented (activated) is poison.
Wait for an archive cycle in this high usage state and the CPU is
still maxed out and my wee_reports fail to cycle. That means that with
my quirky setup it's still running and hasn't finished or been reaped.
Interestingly, when I restart weewx that separate wee_report instance
completes.


On 04/04/2020, Lucas Heijst  wrote:
> Tom, Vince, Glenn,
>
> Glenn,
> Yes, I use external mariadb5 databases.
> And no, during the creation of the vector plots the memory use of weewx is
> not much (2.7 %).
> Changing the aggregate_interval has effect, but is not the main cause.
>
> Tom, Vince, Glenn,
> The main cause of the slow vector calculation is the aggregate_type = max
> on windgustvec.
> Without the max aggregation all 5 vector plots (6h, day, week, month, year)
>
> took together 24 seconds.
>
> Generated 76 images for report vproReport in 113.71 seconds
>
> Luc
>
> =
> [[[hourwindvec]]]
> windvec
> plot_type  = vector
> windgustvec
> plot_type  = vector
> ###aggregate_type = max
> ###aggregate_interval = 900# == 15 min
>
> [[[daywindvec]]]
> windvec
> plot_type= vector
> windgustvec
> plot_type= vector
> ###aggregate_type   = max
> ###aggregate_interval   = 3600# == 1 hour
>
> [[[weekwindvec]]]
> windvec
> plot_type= vector
> windgustvec
> plot_type= vector
> ###aggregate_type   = max
>
> [[[monthwindvec]]]
> windvec
> plot_type= vector
> windgustvec
> plot_type= vector
> ###aggregate_type   = max
> ###aggregate_interval   = 3600# == 1 hour
>
> [[[yearwindvec]]]
> windvec
> plot_type= vector
> windgustvec
> plot_type= vector
> ###aggregate_type   = max
> =
>
>
>
>
> On Friday, 3 April 2020 22:33:50 UTC-3, Glenn McKechnie wrote:
>>
>> (Sigh, and to the list)
>>
>> Luc,
>>
>> I notice a "Launch of report thread aborted: existing report thread
>> still running"
>>
>> Do you use mysql (mariadb)?
>> If you do, does its CPU usage (from top) increase?
>>
>> If so try dropping the aggregate interval = 900 and see if there is a
>> difference
>>
>> On 04/04/2020, Lucas Heijst > wrote:
>> > Tom,
>> >
>> > There was not much info in the syslog, thats why I didnt send it.
>> > I included the syslog this time.
>> >
>> > I was wrong: the reportgenerator didn't hang, only it is VERY slow
>> >
>> > First I generated all other plots: 71 images in 90 seconds
>> > Generated 71 images for report vproReport in 89.58 seconds
>> >
>> > Then I added a 6-hour vector plot. Extra time for that 6h plot: 70
>> seconds
>> > Generated 72 images for report vproReport in 159.32 seconds
>> >
>> > Then added a 24-hour vector plot. Extra time for that 

Re: [weewx-development] Reportgenerator 'hangs' with vector plot

2020-04-04 Thread Thomas Keffer
Luc, could you try the attached version of xtypes.py and see if it makes
any difference?

-tk

On Sat, Apr 4, 2020 at 5:10 AM Lucas Heijst  wrote:

> Tom, Vince, Glenn,
>
> Glenn,
> Yes, I use external mariadb5 databases.
> And no, during the creation of the vector plots the memory use of weewx is
> not much (2.7 %).
> Changing the aggregate_interval has effect, but is not the main cause.
>
> Tom, Vince, Glenn,
> The main cause of the slow vector calculation is the aggregate_type = max
> on windgustvec.
> Without the max aggregation all 5 vector plots (6h, day, week, month,
> year) took together 24 seconds.
>
> Generated 76 images for report vproReport in 113.71 seconds
>
> Luc
>
> =
> [[[hourwindvec]]]
> windvec
> plot_type  = vector
> windgustvec
> plot_type  = vector
> ###aggregate_type = max
> ###aggregate_interval = 900# == 15 min
>
> [[[daywindvec]]]
> windvec
> plot_type= vector
> windgustvec
> plot_type= vector
> ###aggregate_type   = max
> ###aggregate_interval   = 3600# == 1 hour
>
> [[[weekwindvec]]]
> windvec
> plot_type= vector
> windgustvec
> plot_type= vector
> ###aggregate_type   = max
>
> [[[monthwindvec]]]
> windvec
> plot_type= vector
> windgustvec
> plot_type= vector
> ###aggregate_type   = max
> ###aggregate_interval   = 3600# == 1 hour
>
> [[[yearwindvec]]]
> windvec
> plot_type= vector
> windgustvec
> plot_type= vector
> ###aggregate_type   = max
> =
>
>
>
>
> On Friday, 3 April 2020 22:33:50 UTC-3, Glenn McKechnie wrote:
>>
>> (Sigh, and to the list)
>>
>> Luc,
>>
>> I notice a "Launch of report thread aborted: existing report thread
>> still running"
>>
>> Do you use mysql (mariadb)?
>> If you do, does its CPU usage (from top) increase?
>>
>> If so try dropping the aggregate interval = 900 and see if there is a
>> difference
>>
>> On 04/04/2020, Lucas Heijst  wrote:
>> > Tom,
>> >
>> > There was not much info in the syslog, thats why I didnt send it.
>> > I included the syslog this time.
>> >
>> > I was wrong: the reportgenerator didn't hang, only it is VERY slow
>> >
>> > First I generated all other plots: 71 images in 90 seconds
>> > Generated 71 images for report vproReport in 89.58 seconds
>> >
>> > Then I added a 6-hour vector plot. Extra time for that 6h plot: 70
>> seconds
>> > Generated 72 images for report vproReport in 159.32 seconds
>> >
>> > Then added a 24-hour vector plot. Extra time for that 24h vector plot
>> 266
>> > seconds
>> > Generated 73 images for report vproReport in 425.79 seconds
>> >
>> > Estimated time for the other vector plots:
>> > week vector plot 31 minutes
>> > month vector plot: 137 minutes
>> > year vector plot: 27 hours
>> >
>> > Luc
>> >
>> >
>> > On Friday, 3 April 2020 20:13:16 UTC-3, Tom Keffer wrote:
>> >>
>> >> Worked fine for me.
>> >>
>> >> Luc: you know better. We need a log! Perhaps the ReportGenerator is
>> not
>> >> finishing before the next report is due? Perhaps the program
>> segfaulted?
>> >> Who knows without a log?
>> >>
>> >> -tk
>> >> [image: image.png]
>> >>
>> >> On Fri, Apr 3, 2020 at 4:09 PM Lucas Heijst > >> > wrote:
>> >>
>> >>> Currently running weewx 4.0.0b18.
>> >>>
>> >>> The reportgenerator hangs (it never finishes) during calculating of
>> the
>> >>> section below.
>> >>>
>> >>> [[[hourwindvec]]]
>> >>> windvec
>> >>> plot_type  = vector
>> >>> windgustvec
>> >>> plot_type  = vector
>> >>> aggregate_type = max
>> >>> aggregate_interval = 900# == 15 min
>> >>>
>> >>> Luc
>> >>>
>> >>> --
>> >>> You received this message because you are subscribed to the Google
>> Groups
>> >>>
>> >>> "weewx-development" group.
>> >>> To unsubscribe from this group and stop receiving emails from it,
>> send an
>> >>>
>> >>> email to weewx-de...@googlegroups.com .
>> >>> To view this discussion on the web visit
>> >>>
>> https://groups.google.com/d/msgid/weewx-development/601f87df-cbe7-482c-bf49-722a7a5de7bd%40googlegroups.com
>> >>>
>> >>> <
>> https://groups.google.com/d/msgid/weewx-development/601f87df-cbe7-482c-bf49-722a7a5de7bd%40googlegroups.com?utm_medium=email_source=footer>
>>
>> >>> .
>> >>>
>> >>
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups
>> > "weewx-development" group.

Re: [weewx-development] Reportgenerator 'hangs' with vector plot

2020-04-04 Thread Lucas Heijst
Tom, Vince, Glenn,

Glenn,
Yes, I use external mariadb5 databases.
And no, during the creation of the vector plots the memory use of weewx is 
not much (2.7 %).
Changing the aggregate_interval has effect, but is not the main cause.

Tom, Vince, Glenn,
The main cause of the slow vector calculation is the aggregate_type = max 
on windgustvec.
Without the max aggregation all 5 vector plots (6h, day, week, month, year) 
took together 24 seconds.

Generated 76 images for report vproReport in 113.71 seconds

Luc

=
[[[hourwindvec]]]
windvec
plot_type  = vector
windgustvec
plot_type  = vector
###aggregate_type = max
###aggregate_interval = 900# == 15 min

[[[daywindvec]]]
windvec
plot_type= vector
windgustvec
plot_type= vector
###aggregate_type   = max
###aggregate_interval   = 3600# == 1 hour

[[[weekwindvec]]]
windvec
plot_type= vector
windgustvec
plot_type= vector
###aggregate_type   = max

[[[monthwindvec]]]
windvec
plot_type= vector
windgustvec
plot_type= vector
###aggregate_type   = max
###aggregate_interval   = 3600# == 1 hour

[[[yearwindvec]]]
windvec
plot_type= vector
windgustvec
plot_type= vector
###aggregate_type   = max
=




On Friday, 3 April 2020 22:33:50 UTC-3, Glenn McKechnie wrote:
>
> (Sigh, and to the list) 
>
> Luc, 
>
> I notice a "Launch of report thread aborted: existing report thread 
> still running" 
>
> Do you use mysql (mariadb)? 
> If you do, does its CPU usage (from top) increase? 
>
> If so try dropping the aggregate interval = 900 and see if there is a 
> difference 
>
> On 04/04/2020, Lucas Heijst > wrote: 
> > Tom, 
> > 
> > There was not much info in the syslog, thats why I didnt send it. 
> > I included the syslog this time. 
> > 
> > I was wrong: the reportgenerator didn't hang, only it is VERY slow 
> > 
> > First I generated all other plots: 71 images in 90 seconds 
> > Generated 71 images for report vproReport in 89.58 seconds 
> > 
> > Then I added a 6-hour vector plot. Extra time for that 6h plot: 70 
> seconds 
> > Generated 72 images for report vproReport in 159.32 seconds 
> > 
> > Then added a 24-hour vector plot. Extra time for that 24h vector plot 
> 266 
> > seconds 
> > Generated 73 images for report vproReport in 425.79 seconds 
> > 
> > Estimated time for the other vector plots: 
> > week vector plot 31 minutes 
> > month vector plot: 137 minutes 
> > year vector plot: 27 hours 
> > 
> > Luc 
> > 
> > 
> > On Friday, 3 April 2020 20:13:16 UTC-3, Tom Keffer wrote: 
> >> 
> >> Worked fine for me. 
> >> 
> >> Luc: you know better. We need a log! Perhaps the ReportGenerator is not 
> >> finishing before the next report is due? Perhaps the program 
> segfaulted? 
> >> Who knows without a log? 
> >> 
> >> -tk 
> >> [image: image.png] 
> >> 
> >> On Fri, Apr 3, 2020 at 4:09 PM Lucas Heijst  >> > wrote: 
> >> 
> >>> Currently running weewx 4.0.0b18. 
> >>> 
> >>> The reportgenerator hangs (it never finishes) during calculating of 
> the 
> >>> section below. 
> >>> 
> >>> [[[hourwindvec]]] 
> >>> windvec 
> >>> plot_type  = vector 
> >>> windgustvec 
> >>> plot_type  = vector 
> >>> aggregate_type = max 
> >>> aggregate_interval = 900# == 15 min 
> >>> 
> >>> Luc 
> >>> 
> >>> -- 
> >>> You received this message because you are subscribed to the Google 
> Groups 
> >>> 
> >>> "weewx-development" group. 
> >>> To unsubscribe from this group and stop receiving emails from it, send 
> an 
> >>> 
> >>> email to weewx-de...@googlegroups.com . 
> >>> To view this discussion on the web visit 
> >>> 
> https://groups.google.com/d/msgid/weewx-development/601f87df-cbe7-482c-bf49-722a7a5de7bd%40googlegroups.com
>  
> >>> 
> >>> <
> https://groups.google.com/d/msgid/weewx-development/601f87df-cbe7-482c-bf49-722a7a5de7bd%40googlegroups.com?utm_medium=email_source=footer>
>  
>
> >>> . 
> >>> 
> >> 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "weewx-development" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to weewx-de...@googlegroups.com . 
> > To view this discussion on the web visit 
> > 
> 

Re: [weewx-development] weewx-historical

2020-04-04 Thread Thomas Keffer
Silly me. I don't know what made me think this would work with MySQL --- it
does not have an equivalent of strftime().

Let me see what I can do.

-tk

On Sat, Apr 4, 2020 at 3:23 AM Hartmut Schweidler 
wrote:

> Hallo Tom,
>
> Today I installed the Historical extension,  the extension does not work
> with a MYSQL database.
> my Weewx version 4.0.0b18
>
>
> Max Temp 
> $day.outTemp.historical_max
>  im Jahre $day.outTemp.historical_maxtime.format("%Y")
>
> an in syslog
>
> Apr  4 12:21:12 wetter weewx-weewx[10525] DEBUG user.xrainno: MyXRainNo
> SLE executed in 1.774 seconds
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> Generate failed with exception ''
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>  Ignoring template /home/weewx/skins/Basics/trend.html.tmpl
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>  Reason: not enough arguments for format string
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   Traceback (most recent call last):
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 238
> , in execute
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   query = query % args
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   TypeError: not enough arguments for format string
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> 
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   During handling of the above exception, another exception occurred:
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> 
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   Traceback (most recent call last):
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> File "/home/weewx/bin/weedb/mysql.py", line 52, in guarded_fn
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   return fn(*args, **kwargs)
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> File "/home/weewx/bin/weedb/mysql.py", line 262, in execute
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   self.cursor.execute(mysql_string, tuple(sql_tuple))
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 240
> , in execute
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   self.errorhandler(self, ProgrammingError, str(m))
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> File "/usr/lib/python3/dist-packages/MySQLdb/connections.py",
> line 52, in defaulterrorhandler
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   raise errorclass(errorvalue)
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   _mysql_exceptions.ProgrammingError: not enough arguments for format
> string
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> 
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   During handling of the above exception, another exception occurred:
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> 
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   Traceback (most recent call last):
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> File "/home/weewx/bin/weewx/cheetahgenerator.py", line 323, in
> generate
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   unicode_string = compiled_template.respond()
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> File "_home_weewx_skins_Basics_trend_html_tmpl.py", line 422, in
> respond
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> File "/usr/lib/python3/dist-packages/Cheetah/Template.py", line
> 1707, in _handleCheetahInclude
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   self._CHEETAH__cheetahIncludes[_includeID].respond(trans)
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> File
> "cheetah__home_weewx_skins_Standard_hes_current_inc_1585994730_8648148_88513.py"
> , line 114, in respond
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> File "/home/weewx/bin/weewx/tags.py", line 347, in __getattr__
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
>   return self._do_query(aggregate_type)
> Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:
> File 

Re: [weewx-development] weewx-historical

2020-04-04 Thread Hartmut Schweidler
Hallo Tom,

Today I installed the Historical extension,  the extension does not work 
with a MYSQL database.
my Weewx version 4.0.0b18

   
Max Temp 
$day.outTemp.historical_max
 im Jahre $day.outTemp.historical_maxtime.format("%Y")
   
an in syslog

Apr  4 12:21:12 wetter weewx-weewx[10525] DEBUG user.xrainno: MyXRainNo SLE 
executed in 1.774 seconds
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator: 
Generate failed with exception ''
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
Ignoring template /home/weewx/skins/Basics/trend.html.tmpl
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
Reason: not enough arguments for format string
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 Traceback (most recent call last):
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
   File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 238, in 
execute
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 query = query % args
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 TypeError: not enough arguments for format string
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 During handling of the above exception, another exception occurred:
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 Traceback (most recent call last):
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
   File "/home/weewx/bin/weedb/mysql.py", line 52, in guarded_fn
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 return fn(*args, **kwargs)
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
   File "/home/weewx/bin/weedb/mysql.py", line 262, in execute
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 self.cursor.execute(mysql_string, tuple(sql_tuple))
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
   File "/usr/lib/python3/dist-packages/MySQLdb/cursors.py", line 240, in 
execute
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 self.errorhandler(self, ProgrammingError, str(m))
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
   File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 52, in 
defaulterrorhandler
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 raise errorclass(errorvalue)
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 _mysql_exceptions.ProgrammingError: not enough arguments for format string
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 During handling of the above exception, another exception occurred:
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 Traceback (most recent call last):
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
   File "/home/weewx/bin/weewx/cheetahgenerator.py", line 323, in generate
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 unicode_string = compiled_template.respond()
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
   File "_home_weewx_skins_Basics_trend_html_tmpl.py", line 422, in respond
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
   File "/usr/lib/python3/dist-packages/Cheetah/Template.py", line 1707, in 
_handleCheetahInclude
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 self._CHEETAH__cheetahIncludes[_includeID].respond(trans)
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
   File 
"cheetah__home_weewx_skins_Standard_hes_current_inc_1585994730_8648148_88513.py"
, line 114, in respond
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
   File "/home/weewx/bin/weewx/tags.py", line 347, in __getattr__
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 return self._do_query(aggregate_type)
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
   File "/home/weewx/bin/weewx/tags.py", line 364, in _do_query
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
 db_manager, val=val, **self.option_dict)
Apr  4 12:21:12 wetter weewx-weewx[10525] ERROR weewx.cheetahgenerator:  
   File "/home/weewx/bin/weewx/xtypes.py", line 102, in get_aggregate
Apr  4 12:21:12