If it's on a remote server, it can still be a problem. A tag like $day.outTemp.max involves a simple database lookup. A typical skin can have hundreds of such tags, resulting in hundreds of queries. SQLite excels at these kinds of frequent small queries, MySQL does not.
Try running with sqlite and see if it makes a difference. It's just a few options in your configuration file to give it a go. -tk On Fri, Feb 5, 2021 at 1:34 PM Tim Lambert <[email protected]> wrote: > Tom, > > > > MySQL is on a remote/hosted server – it is not running on the Pi. I’ve > provided the connection parameters in the weewx.conf > > > > > > > > Regards, > > > > Tim > > > > *From: *Tom Keffer <[email protected]> > *Sent: *Friday, February 5, 2021 1:31 PM > *To: *weewx-user <[email protected]> > *Subject: *Re: [weewx-user] Re: WEEWX-WD and Weewx 4.2 to 4.4 > > > > Your machine should be more than capable of running weewx. > > > > However, your weak spot is probably MySQL, particularly if you are running > it on an SD card. MySQL is *much* slower than sqlite. > > > > See the wiki article *SQLite vs MySQL > <https://github.com/weewx/weewx/wiki/SQLite-vs-MySQL>*. > > > > On Fri, Feb 5, 2021 at 12:55 PM vince <[email protected]> wrote: > > You have something major happening on that pi of yours. Here's some data > from a clean RaspiOS and newly installed weewx running the Simulator driver > on a 4GB pi4... > > > > Basically no RAM usage.... > > > > root@pi4p1:/home/weewx# free > > total used free shared buff/cache > available > > Mem: 3919788 72836 3516564 8620 330388 > 3705176 > > Swap: 102396 0 102396 > > > > And no CPU usage.... > > > > root@pi4p1:/home/weewx# uptime > > 11:50:49 up 40 min, 1 user, load average: 0.00, 0.00, 0.01 > > > > Logs for the first couple cycles - the initial time for a skin is always > longer than the normal time after that.... > > > > Feb 5 11:30:46 pi4p1 weewx[1839] INFO __main__: Starting up weewx version > 4.4.0 > > Feb 5 11:30:46 pi4p1 weewx[1839] INFO weewx.engine: Clock error is -0.09 > seconds (positive is fast) > > Feb 5 11:30:46 pi4p1 weewx[1839] INFO weewx.engine: Using binding > 'wx_binding' to database 'weewx.sdb' > > Feb 5 11:30:46 pi4p1 weewx[1839] INFO weewx.manager: Starting backfill of > daily summaries > > Feb 5 11:30:46 pi4p1 weewx[1839] INFO weewx.manager: Empty database > > Feb 5 11:30:46 pi4p1 weewx[1839] INFO weewx.engine: Starting main packet > loop. > > Feb 5 11:35:16 pi4p1 weewx[1839] INFO weewx.manager: Added record > 2021-02-05 11:35:00 AKST (1612557300) to database 'weewx.sdb' > > Feb 5 11:35:16 pi4p1 weewx[1839] INFO weewx.manager: Added record > 2021-02-05 11:35:00 AKST (1612557300) to daily summary in 'weewx.sdb' > > Feb 5 11:35:18 pi4p1 weewx[1839] INFO weewx.cheetahgenerator: Generated 8 > files for report SeasonsReport in 1.79 seconds > > Feb 5 11:35:25 pi4p1 weewx[1839] INFO weewx.imagegenerator: Generated 60 > images for report SeasonsReport in 6.35 seconds > > Feb 5 11:35:25 pi4p1 weewx[1839] INFO weewx.reportengine: Copied 5 files > to /home/weewx/public_html > > Feb 5 11:40:16 pi4p1 weewx[1839] INFO weewx.manager: Added record > 2021-02-05 11:40:00 AKST (1612557600) to database 'weewx.sdb' > > Feb 5 11:40:16 pi4p1 weewx[1839] INFO weewx.manager: Added record > 2021-02-05 11:40:00 AKST (1612557600) to daily summary in 'weewx.sdb' > > Feb 5 11:40:17 pi4p1 weewx[1839] INFO weewx.cheetahgenerator: Generated 8 > files for report SeasonsReport in 0.80 seconds > > Feb 5 11:40:18 pi4p1 weewx[1839] INFO weewx.imagegenerator: Generated 15 > images for report SeasonsReport in 0.51 seconds > > > > On Friday, February 5, 2021 at 11:29:56 AM UTC-8 tim lambert wrote: > > Tom, > > > > Is the issue CPU or RAM? > > > > I’m running a Pi 4 with 4gb of RAM. > > > > Regards, > > > > Tim > > > > Sent from my Hi-Tech Etch-A-Sketch > > > > > > On Feb 5, 2021, at 11:11, Tom Keffer <[email protected]> wrote: > > > > The problem is that the machine running weewx doesn't have enough "oomph" > to run all the reports you are asking of it in two minutes. For example, > here's the results of one reporting cycle: > > > > Feb 5 09:00:23 murph weewx[15605] INFO weewx.cheetahgenerator: Generated > 8 files for report SeasonsReport in 95.95 seconds > Feb 5 09:00:31 murph weewx[15605] INFO weewx.imagegenerator: Generated 15 > images for report SeasonsReport in 7.99 seconds > Feb 5 09:00:32 murph weewx[15605] INFO user.stackedwindrose: Generated 1 > images for wdStackedWindRose in 1.11 seconds > Feb 5 09:02:05 murph weewx[15605] INFO weewx.cheetahgenerator: Generated > 1 files for report wdTesttags in 93.15 seconds > Feb 5 09:02:53 murph weewx[15605] INFO weewx.cheetahgenerator: Generated > 2 files for report wdClientraw in 47.34 seconds > Feb 5 09:03:19 murph weewx[15605] INFO weewx.cheetahgenerator: Generated > 2 files for report SteelSeries in 8.65 seconds > Feb 5 09:03:25 murph weewx[15605] INFO weewx.imagegenerator: Generated 11 > images for report SteelSeries in 5.64 seconds > > > > That adds up to almost 300 seconds --- way more than the 120 seconds you > are giving it. You will either have to go to a much longer reporting cycle > (5 minutes may not even do it), or pare down the number of reports, or > simplify the reports, or get a more capable server. > > > > -tk > > > > > > > > > > > > On Fri, Feb 5, 2021 at 10:11 AM Tim Lambert <[email protected]> wrote: > > Tom, > > > > Attached is the logfile for 2 cycles. > > > > Per Line 45 @ 08:56:54 – The 120sec Interval for wwewx.engine is loaded > > Per Line 137 – Line 141 @ 08:58:33 – The Remote MySQL DB is being updated > > Per Line 143 @ 08:58:47 – The rpt.engine starts > > Per Line 218 @ 09:00:40 – Error weewx.engine unable to shutdown StdReport > thread > > Then it appears weew opted to perform a restart by itself > > Per Line 333 – Line 337 @ 09:02:32 – The Remote MySQL DB is being updated > > Per Line 353 @ 09:02:53 – Start FTP > > Per Line 421 @ 09:03:11 – End FTP > > Per Line 423 @ 09:03:11 – Run SS > > Per Line 471 @ 09:04:22 – Start Reports > > Per Line 487 – Line 491 – The Remote MySQL DB is being updated > > Per Line 492 @ 09:04:48 – Report Thread is aborted, existing Report Thread > is running > > Per Line 552 @ 09:06:40– Error weewx.engine unable to shutdown StdReport > thread > > Then it appears weew opted to perform a restart by itself > > Per Line 653 @ 09:08:05 – Start FTP > > Per Line 716 @ 09:08:23 – End FTP > > Per Line 723 @ 09:08:32 – Run SS > > > > Hope the above analysis helps pinpoint the problem. > > > > I’m concerned about the Error related to the shutdown of the StdReport > Thread and the subsequent automatic restart. My other concern is the > aborting the Report Thread, because the existing Report Thread is still > running. Both appear to be impacting the 120 second interval to generate > reports and subsequently FTP, as well as the write of the DB every 120 > seconds. > > > > > > Thanks in Advance for your help. > > > > > > Regards, > > > > Tim > > > > *From: *Tom Keffer > *Sent: *Thursday, February 4, 2021 6:20 PM > *To: *weewx-user > *Subject: *Re: [weewx-user] Re: WEEWX-WD and Weewx 4.2 to 4.4 > > > > That station (using the ip100 driver) does not support hardware record > generation, so WeeWX will fall back on software record generation. If you > are still getting 5 minute updates despite setting archive_interval to 120, > then one of two things is happening: > > 1. weewxd is using a different configuration file than you think it > is; or > > 2. You didn't restart weewxd after changing the value. > > If neither of those are true, then we will need to see the log. Set > debug=1, restart weewx, post the log from startup through the first > reporting cycle. > > > > tk > > > > On Thu, Feb 4, 2021 at 5:41 PM Tim <[email protected]> wrote: > > Tom, > > > > The station is a RainWise MK-III with an IP-100. > > Regards, > > > > Tim > > > > Sent from my Hi-Tech Etch-A-Sketch > > > > > > On Feb 4, 2021, at 14:20, Tom Keffer <[email protected]> wrote: > > > > You didn't say what kind of weather station you are using, but for > stations that support hardware logging and if you specify hardware record > generation, then weewx will use the interval specified in your hardware. > > > > If this is the case, then to change use the wee_device utility with the > --set-interval option. For example, to change to two minutes: > > > > *wee_device --set-interval=2* > > > > -tk > > > > > > > > On Thu, Feb 4, 2021 at 9:12 AM tim lambert <[email protected]> wrote: > > Hi Gary, > > > > I'm taking things in baby steps. > > > > Earlier this week I successfully upgraded from weewx 4.2 to weewx 4.4 > (which was surprisingly easy with how weewx addresses upgrades). > Everything is stable, so I'm moving onward to the next steps. > > > > I've taken the route to implement WeeWX SteelSeries Weather Gauges > extension (not real-time yet) and WeeWX-WD v2.1.2 -- everything installed > and I can see the SteelGauge info on my remotely hosted website, as well as > see the clientraw.text and other key files for Saratoga Templates on the > remotely hosted server. > > > > However, even though I have the archive_interval in weewx.conf set to 120 > (seconds) the FTP of the files generated by weewx is occurring every 300 > seconds (5 minutes) -- which I confirmed from reviewing the log files. > How can I get the FTP to execute every 120 seconds? > > > > FYI, weewx.conf for Standard Archive is > > > > > > > > > *[StdArchive] archive_interval = 120 record_generation = hardware > loop_hilo = True data_binding = wx_binding* > > > > Should I change the record_generation to software? Noting the Rainwise > IP-100 is inserting records into the MySQL database tables every minute -- > which is what I want. > > > > Thanks in Advance for your guidance! > > > > -- Tim > > > > On Saturday, January 30, 2021 at 8:35:31 PM UTC-8 tim lambert wrote: > > Thanks Gary for the insight and guidance. > > > > > > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for > Windows 10 > > > > *From: *gjr80 > *Sent: *Saturday, January 30, 2021 7:58 PM > *To: *weewx-user > *Subject: *[weewx-user] Re: WEEWX-WD and Weewx 4.2 to 4.4 > > > > Hi, > > > > I wouldn't be using WeeWX-WD from bitbucket, it has not been touched for > some years now. WeeWX-WD on github > <https://github.com/gjr80/weewx-weewx-wd> is more up to date, the current > version is v2.1.2 from November 2020. It should have no problems running on > WeeWX v4.x under python 2 or python3. The SteelSeries Weather Gauges > <https://github.com/mcrossley/SteelSeries-Weather-Gauges> have directly > supported WeeWX of v2.5.0 (of the SteelSeries Weather Gauges) or for about > six years, and there is now a WeeWX SteelSeries Gauges extension > <https://github.com/weewx/weewx/wiki/steelseries> that installs the > necessary skin to support the SteelSeries Weather Gauges (this is the > preferred means of providing the SteelSeries Weather Gauges, WeeWX-WD did > support the SteelSeries Weather Gauges via the WD customclientraw.txt but > since the advent of direct WeeWX support and the WeeWX extension the > SteelSeries Weather Gauges support has been deprecated in WeeWX-WD). You > should find the WeeWX extension will work under WeeWX v4.x and python2 or > python3. > > > > I would suggest that you upgrade to WeeWX v4.4.0 and get WeeWX running to > your satisfaction before installing either WeeWX-WD v2.1.2 or the WeeWX > SteelSeries Weather Gauges extension (in whatever order suits). Once you > have the SteelSeries Weather Gauges extension running OK you can, if you > want, look at installing the Realtime gauge-data extension > <https://github.com/gjr80/weewx-realtime_gauge-data> which will allow the > gauges to update on every loop packet rather than on each report cycle (how > often they will update will depend on how often your station emits loop > packets, for your Rainwise this should be up to every couple of seconds). > > > > Gary > > > > On Sunday, 31 January 2021 at 13:15:24 UTC+10 tim lambert wrote: > > Gang, > > > > Currently running weewx 4.2 (will be upgrading in the next few days to > 4.4) on a Raspberry Pi 4 (32-bit OS) with a Rainwise MK-III and IP-100 > providing station data -- everything has been working properly for several > weeks and is populating a remote MySQL DB. > > > > Once I upgrade weewx, I'd like to incorporate using Steel Gauges and > Saratoga Templates, hence my query: Can I utilize the WEEWX-WD as outlined > BitBucket > Weewx-WD <https://bitbucket.org/ozgreg/weewx-wd/wiki/Users%20Guide>? > > > > The Steel Gauges and Saratoga Templates will be hosted on a remote, public > accessible server -- so I'll be sending the clientraw and testtag files via > FTP. > > > > Thanks in Advance! > > > > - Tim > > -- > You received this message because you are subscribed to a topic in the > Google Groups "weewx-user" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/weewx-user/gvReyZeqOJQ/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/6942d0aa-94bf-46e9-bb56-8232a425f95fn%40googlegroups.com > <https://groups.google.com/d/msgid/weewx-user/6942d0aa-94bf-46e9-bb56-8232a425f95fn%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/e8c94cec-3d75-4ac1-93d3-19cfcae8a1b6n%40googlegroups.com > <https://groups.google.com/d/msgid/weewx-user/e8c94cec-3d75-4ac1-93d3-19cfcae8a1b6n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to a topic in the > Google Groups "weewx-user" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/weewx-user/gvReyZeqOJQ/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/CAPq0zEC0JotC0Rrp0NirvcUx8%3DvL6kzY9W7%3D1SbdU75_ke1i%3DQ%40mail.gmail.com > <https://groups.google.com/d/msgid/weewx-user/CAPq0zEC0JotC0Rrp0NirvcUx8%3DvL6kzY9W7%3D1SbdU75_ke1i%3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/ECA9484A-7708-449E-B0C1-DDB22A0F9DEA%40comcast.net > <https://groups.google.com/d/msgid/weewx-user/ECA9484A-7708-449E-B0C1-DDB22A0F9DEA%40comcast.net?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to a topic in the > Google Groups "weewx-user" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/weewx-user/gvReyZeqOJQ/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/CAPq0zEAK_512pG%3DDPbghbVCnJSJQjk2Q9uEHW12g4WEZkP4CTw%40mail.gmail.com > <https://groups.google.com/d/msgid/weewx-user/CAPq0zEAK_512pG%3DDPbghbVCnJSJQjk2Q9uEHW12g4WEZkP4CTw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/601d8a3f.1c69fb81.c2a39.613cSMTPIN_ADDED_MISSING%40gmr-mx.google.com > <https://groups.google.com/d/msgid/weewx-user/601d8a3f.1c69fb81.c2a39.613cSMTPIN_ADDED_MISSING%40gmr-mx.google.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to a topic in the > Google Groups "weewx-user" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/weewx-user/gvReyZeqOJQ/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/CAPq0zEB9Fcu3tLZpGVFqhTqoQjXYzLgMBP-R5vjRqBx-jyXVQw%40mail.gmail.com > <https://groups.google.com/d/msgid/weewx-user/CAPq0zEB9Fcu3tLZpGVFqhTqoQjXYzLgMBP-R5vjRqBx-jyXVQw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/7a3c3f02-37e0-4ba1-9a45-d45797eee986n%40googlegroups.com > <https://groups.google.com/d/msgid/weewx-user/7a3c3f02-37e0-4ba1-9a45-d45797eee986n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to a topic in the > Google Groups "weewx-user" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/weewx-user/gvReyZeqOJQ/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/CAPq0zEAKnx3HS_8g-OBz%2BoAkNu9ky-Z6e1xp9DD8HNFGwmxeZg%40mail.gmail.com > <https://groups.google.com/d/msgid/weewx-user/CAPq0zEAKnx3HS_8g-OBz%2BoAkNu9ky-Z6e1xp9DD8HNFGwmxeZg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/601db9f5.1c69fb81.d7df7.95d2SMTPIN_ADDED_MISSING%40gmr-mx.google.com > <https://groups.google.com/d/msgid/weewx-user/601db9f5.1c69fb81.d7df7.95d2SMTPIN_ADDED_MISSING%40gmr-mx.google.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEDFTs2_yBFKbmmw7hwbEo6GLQup%2ByYJF6BZvKeGX4ub0g%40mail.gmail.com.
