Team, A quick update...
I've completed standing up WeeWx 4.4 on a dedicated (for now) Pi 4, with the IP-100 extension using the Seasons Skins and FTP the Seasons reports/graphs to a remote public server. The performance stats are: *Seasons Reports == 1.21 seconds* *Season Images == 0.54 seconds* *FTP == 6.89 seconds* *Total Processing Time == 8.64 seconds* Next I added in writing to to a MariaDB on a 1TB hard drive connected directly to the Pi4. The aforementioned performance stats did not change. Next, I implemented the WD Extension, using the local MariaDB to write the databases suggested by WD Extensions. The performance stats: *Seasons Reports == 1.21 seconds* *Seasons Images == 0.60 seconds* *WD Files (testtags, windrose & clientraw files) == 1.13 seconds* *FTP == 8.26 seconds* *Total Processing Time == 11.20 seconds* Finally, I added in the Steel Gauges, which the average performance stats: *Seasons Reports == 2.42 seconds* *Seasons Images == 0.63 seconds* * WD Files (testtags, windrose & clientraw files) == 4.27 seconds * *Steel Gauges == 0.74 seconds* *FTP == 16.02 seconds* *Total Processing Time == 24.08 seconds* Note: the only other delta between the original configuration and the new configuration, besides moving the MySQL/MariaDB from a remote/hosted server to a local server with an HD -- the original configuration also had consoleWD from Weather Display. Conclusion -- the data exchange between weeWX and the remote/hosted server was extremely intensive and thus slowing down the reporting process (which Gary and Tom both thought was the root cause). Thanks for helping me sleuth this one -- hopefully others will benefit from the errors in my architecture. -- Tim On Friday, February 5, 2021 at 10:57:47 PM UTC-8 tim lambert wrote: > Thanks everyone for your input... You have all pointed me to what is > beginning to look like the root cause -- using a remote MySQL server. > > So far, I've stood up another Pi 4 to be dedicated for weewx (for now). A > basic installation using the Seasons skin -- reports/graphs are generating > in roughly 1.2 seconds. FTP of the public_html is taking 7 seconds. > > Tomorrow, I'll stand up Maria DB (Raspberry version of MySQL) on the same > Pi, with the DB being on a 1TB attached drive. Once Maria DB is installed > and configured, I'll add in phpMyAdmin (to help with quick DB Table > analysis). Next will be to install WD Extensions and SS Gauges, which > will utilize the DB on the attached drive. > > Hoping for a marked performance improvement -- I'll share my learnings. > > -- Tim > > On Friday, February 5, 2021 at 6:07:50 PM UTC-8 vince wrote: > >> Just for kicks, run "*top*", hit '*1*' to see all the cpu cores >> individually, and see if you see anything with a percentage other than 0.0 >> next to "*ni*" in the output. That'll help us see if you have things >> waiting for i/o. >> >> You might need to let it run for a little to watch things start/stop and >> change states on the various cpus, but concentrate on the '*ni*' item in >> each line there. To get out of top, hit '*q*' for quit. >> >> My pi3 reports the following: >> >> top - 18:04:58 up 9 days, 30 min, 3 users, load average: 0.12, 0.15, >> 0.12 >> Tasks: 137 total, 2 running, 135 sleeping, 0 stopped, 0 zombie >> %Cpu0 : 4.0 us, 0.3 sy, 0.0 ni, 95.6 id, 0.0 wa, 0.0 hi, 0.0 si, >> 0.0 st >> %Cpu1 : 1.0 us, 0.0 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.0 hi, 0.0 si, >> 0.0 st >> %Cpu2 : 1.7 us, 0.0 sy, 0.0 ni, 98.3 id, 0.0 wa, 0.0 hi, 0.0 si, >> 0.0 st >> %Cpu3 : 0.3 us, 1.0 sy, 0.0 ni, 98.6 id, 0.0 wa, 0.0 hi, 0.0 si, >> 0.0 st >> MiB Mem : 926.1 total, 30.5 free, 349.4 used, 546.2 >> buff/cache >> MiB Swap: 100.0 total, 98.5 free, 1.5 used. 443.5 avail Mem >> >> On Friday, February 5, 2021 at 5:32:33 PM UTC-8 tim lambert wrote: >> >>> Gary, >>> >>> >>> >>> I think you hit on the root cause. >>> >>> >>> >>> I’m standing up another Pi with an external HD attached to support the >>> DB – the idea is to keep the DB traffic off the SD card. >>> >>> >>> >>> Regards, >>> >>> >>> >>> Tim >>> >>> >>> >>> *From: *gjr80 >>> *Sent: *Friday, February 5, 2021 2:42 PM >>> *To: *weewx-user >>> *Subject: *Re: [weewx-user] Re: WEEWX-WD and Weewx 4.2 to 4.4 >>> >>> >>> >>> I’m not sure MySQL is the issue per se, rather I suspect the remote >>> connection. The main WeeWX-WD template testtags.php.tmpl has some 1000 odd >>> $day, $month, $year tags; of the other four clientraw templates there is >>> some 500 $day, $month, $year tags And we haven’t even gotten to the search >>> list extensions of which there are numerous and some quite database >>> intensive. I run WeeWX-WD on a RPi 3B+ along with about 10 other skins and >>> and my total report generation time is just over 45 seconds. That is using >>> MySQL locally and running a second instance of WeeWX-WD to support a GW1000. >>> >>> >>> >>> I recall in the syslog extract extract there was a lost connection to >>> the MySQL server, seems to me to point to some issues accessing the remote >>> database server. >>> >>> >>> >>> Gary >>> >>> On Saturday, 6 February 2021 at 08:18:51 UTC+10 [email protected] wrote: >>> >>> If it's on a remote server, it can still be a problem. >>> >>> >>> >>> A tag like $day.outTemp.max involves a simple database lookup. A typical >>> skin can have hundreds of such tags, resulting in hundreds of queries. >>> SQLite excels at these kinds of frequent small queries, MySQL does not. >>> >>> >>> >>> Try running with sqlite and see if it makes a difference. It's just a >>> few options in your configuration file to give it a go. >>> >>> >>> >>> -tk >>> >>> >>> >>> On Fri, Feb 5, 2021 at 1:34 PM Tim Lambert <[email protected]> wrote: >>> >>> Tom, >>> >>> >>> >>> MySQL is on a remote/hosted server – it is not running on the Pi. I’ve >>> provided the connection parameters in the weewx.conf >>> >>> >>> >>> >>> >>> >>> >>> Regards, >>> >>> >>> >>> Tim >>> >>> >>> >>> *From: *Tom Keffer >>> *Sent: *Friday, February 5, 2021 1:31 PM >>> *To: *weewx-user >>> *Subject: *Re: [weewx-user] Re: WEEWX-WD and Weewx 4.2 to 4.4 >>> >>> >>> >>> Your machine should be more than capable of running weewx. >>> >>> >>> >>> However, your weak spot is probably MySQL, particularly if you are >>> running it on an SD card. MySQL is *much* slower than sqlite. >>> >>> >>> >>> See the wiki article *SQLite vs MySQL >>> <https://github.com/weewx/weewx/wiki/SQLite-vs-MySQL>*. >>> >>> >>> >>> On Fri, Feb 5, 2021 at 12:55 PM vince <[email protected]> wrote: >>> >>> You have something major happening on that pi of yours. Here's some data >>> from a clean RaspiOS and newly installed weewx running the Simulator driver >>> on a 4GB pi4... >>> >>> >>> >>> Basically no RAM usage.... >>> >>> >>> >>> root@pi4p1:/home/weewx# free >>> >>> total used free shared buff/cache >>> available >>> >>> Mem: 3919788 72836 3516564 8620 330388 >>> 3705176 >>> >>> Swap: 102396 0 102396 >>> >>> >>> >>> And no CPU usage.... >>> >>> >>> >>> root@pi4p1:/home/weewx# uptime >>> >>> 11:50:49 up 40 min, 1 user, load average: 0.00, 0.00, 0.01 >>> >>> >>> >>> Logs for the first couple cycles - the initial time for a skin is always >>> longer than the normal time after that.... >>> >>> >>> >>> Feb 5 11:30:46 pi4p1 weewx[1839] INFO __main__: Starting up weewx >>> version 4.4.0 >>> >>> Feb 5 11:30:46 pi4p1 weewx[1839] INFO weewx.engine: Clock error is >>> -0.09 seconds (positive is fast) >>> >>> Feb 5 11:30:46 pi4p1 weewx[1839] INFO weewx.engine: Using binding >>> 'wx_binding' to database 'weewx.sdb' >>> >>> Feb 5 11:30:46 pi4p1 weewx[1839] INFO weewx.manager: Starting backfill >>> of daily summaries >>> >>> Feb 5 11:30:46 pi4p1 weewx[1839] INFO weewx.manager: Empty database >>> >>> Feb 5 11:30:46 pi4p1 weewx[1839] INFO weewx.engine: Starting main >>> packet loop. >>> >>> Feb 5 11:35:16 pi4p1 weewx[1839] INFO weewx.manager: Added record >>> 2021-02-05 11:35:00 AKST (1612557300) to database 'weewx.sdb' >>> >>> Feb 5 11:35:16 pi4p1 weewx[1839] INFO weewx.manager: Added record >>> 2021-02-05 11:35:00 AKST (1612557300) to daily summary in 'weewx.sdb' >>> >>> Feb 5 11:35:18 pi4p1 weewx[1839] INFO weewx.cheetahgenerator: Generated >>> 8 files for report SeasonsReport in 1.79 seconds >>> >>> Feb 5 11:35:25 pi4p1 weewx[1839] INFO weewx.imagegenerator: Generated >>> 60 images for report SeasonsReport in 6.35 seconds >>> >>> Feb 5 11:35:25 pi4p1 weewx[1839] INFO weewx.reportengine: Copied 5 >>> files to /home/weewx/public_html >>> >>> Feb 5 11:40:16 pi4p1 weewx[1839] INFO weewx.manager: Added record >>> 2021-02-05 11:40:00 AKST (1612557600) to database 'weewx.sdb' >>> >>> Feb 5 11:40:16 pi4p1 weewx[1839] INFO weewx.manager: Added record >>> 2021-02-05 11:40:00 AKST (1612557600) to daily summary in 'weewx.sdb' >>> >>> Feb 5 11:40:17 pi4p1 weewx[1839] INFO weewx.cheetahgenerator: Generated >>> 8 files for report SeasonsReport in 0.80 seconds >>> >>> Feb 5 11:40:18 pi4p1 weewx[1839] INFO weewx.imagegenerator: Generated >>> 15 images for report SeasonsReport in 0.51 seconds >>> >>> >>> >>> On Friday, February 5, 2021 at 11:29:56 AM UTC-8 tim lambert wrote: >>> >>> Tom, >>> >>> >>> >>> Is the issue CPU or RAM? >>> >>> >>> >>> I’m running a Pi 4 with 4gb of RAM. >>> >>> >>> >>> Regards, >>> >>> >>> >>> Tim >>> >>> >>> >>> Sent from my Hi-Tech Etch-A-Sketch >>> >>> >>> >>> >>> >>> On Feb 5, 2021, at 11:11, Tom Keffer <[email protected]> wrote: >>> >>> >>> >>> The problem is that the machine running weewx doesn't have enough >>> "oomph" to run all the reports you are asking of it in two minutes. For >>> example, here's the results of one reporting cycle: >>> >>> >>> >>> Feb 5 09:00:23 murph weewx[15605] INFO weewx.cheetahgenerator: >>> Generated 8 files for report SeasonsReport in 95.95 seconds >>> Feb 5 09:00:31 murph weewx[15605] INFO weewx.imagegenerator: Generated >>> 15 images for report SeasonsReport in 7.99 seconds >>> Feb 5 09:00:32 murph weewx[15605] INFO user.stackedwindrose: Generated >>> 1 images for wdStackedWindRose in 1.11 seconds >>> Feb 5 09:02:05 murph weewx[15605] INFO weewx.cheetahgenerator: >>> Generated 1 files for report wdTesttags in 93.15 seconds >>> Feb 5 09:02:53 murph weewx[15605] INFO weewx.cheetahgenerator: >>> Generated 2 files for report wdClientraw in 47.34 seconds >>> Feb 5 09:03:19 murph weewx[15605] INFO weewx.cheetahgenerator: >>> Generated 2 files for report SteelSeries in 8.65 seconds >>> Feb 5 09:03:25 murph weewx[15605] INFO weewx.imagegenerator: Generated >>> 11 images for report SteelSeries in 5.64 seconds >>> >>> >>> >>> That adds up to almost 300 seconds --- way more than the 120 seconds you >>> are giving it. You will either have to go to a much longer reporting cycle >>> (5 minutes may not even do it), or pare down the number of reports, or >>> simplify the reports, or get a more capable server. >>> >>> >>> >>> -tk >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> On Fri, Feb 5, 2021 at 10:11 AM Tim Lambert <[email protected]> >>> wrote: >>> >>> Tom, >>> >>> >>> >>> Attached is the logfile for 2 cycles. >>> >>> >>> >>> Per Line 45 @ 08:56:54 – The 120sec Interval for wwewx.engine is loaded >>> >>> Per Line 137 – Line 141 @ 08:58:33 – The Remote MySQL DB is being >>> updated >>> >>> Per Line 143 @ 08:58:47 – The rpt.engine starts >>> >>> Per Line 218 @ 09:00:40 – Error weewx.engine unable to shutdown >>> StdReport thread >>> >>> Then it appears weew opted to perform a restart by itself >>> >>> Per Line 333 – Line 337 @ 09:02:32 – The Remote MySQL DB is being updated >>> >>> Per Line 353 @ 09:02:53 – Start FTP >>> >>> Per Line 421 @ 09:03:11 – End FTP >>> >>> Per Line 423 @ 09:03:11 – Run SS >>> >>> Per Line 471 @ 09:04:22 – Start Reports >>> >>> Per Line 487 – Line 491 – The Remote MySQL DB is being updated >>> >>> Per Line 492 @ 09:04:48 – Report Thread is aborted, existing Report >>> Thread is running >>> >>> Per Line 552 @ 09:06:40– Error weewx.engine unable to shutdown >>> StdReport thread >>> >>> Then it appears weew opted to perform a restart by itself >>> >>> Per Line 653 @ 09:08:05 – Start FTP >>> >>> Per Line 716 @ 09:08:23 – End FTP >>> >>> Per Line 723 @ 09:08:32 – Run SS >>> >>> >>> >>> Hope the above analysis helps pinpoint the problem. >>> >>> >>> >>> I’m concerned about the Error related to the shutdown of the StdReport >>> Thread and the subsequent automatic restart. My other concern is the >>> aborting the Report Thread, because the existing Report Thread is still >>> running. Both appear to be impacting the 120 second interval to generate >>> reports and subsequently FTP, as well as the write of the DB every 120 >>> seconds. >>> >>> >>> >>> >>> >>> Thanks in Advance for your help. >>> >>> >>> >>> >>> >>> Regards, >>> >>> >>> >>> Tim >>> >>> >>> >>> *From: *Tom Keffer >>> *Sent: *Thursday, February 4, 2021 6:20 PM >>> *To: *weewx-user >>> *Subject: *Re: [weewx-user] Re: WEEWX-WD and Weewx 4.2 to 4.4 >>> >>> >>> >>> That station (using the ip100 driver) does not support hardware record >>> generation, so WeeWX will fall back on software record generation. If you >>> are still getting 5 minute updates despite setting archive_interval to 120, >>> then one of two things is happening: >>> >>> 1. weewxd is using a different configuration file than you think >>> it is; or >>> >>> 2. You didn't restart weewxd after changing the value. >>> >>> If neither of those are true, then we will need to see the log. Set >>> debug=1, restart weewx, post the log from startup through the first >>> reporting cycle. >>> >>> >>> >>> tk >>> >>> >>> >>> On Thu, Feb 4, 2021 at 5:41 PM Tim <[email protected]> wrote: >>> >>> Tom, >>> >>> >>> >>> The station is a RainWise MK-III with an IP-100. >>> >>> Regards, >>> >>> >>> >>> Tim >>> >>> >>> >>> Sent from my Hi-Tech Etch-A-Sketch >>> >>> >>> >>> >>> >>> On Feb 4, 2021, at 14:20, Tom Keffer <[email protected]> wrote: >>> >>> >>> >>> You didn't say what kind of weather station you are using, but for >>> stations that support hardware logging and if you specify hardware record >>> generation, then weewx will use the interval specified in your hardware. >>> >>> >>> >>> If this is the case, then to change use the wee_device utility with the >>> --set-interval option. For example, to change to two minutes: >>> >>> >>> >>> *wee_device --set-interval=2* >>> >>> >>> >>> -tk >>> >>> >>> >>> >>> >>> >>> >>> On Thu, Feb 4, 2021 at 9:12 AM tim lambert <[email protected]> wrote: >>> >>> Hi Gary, >>> >>> >>> >>> I'm taking things in baby steps. >>> >>> >>> >>> Earlier this week I successfully upgraded from weewx 4.2 to weewx 4.4 >>> (which was surprisingly easy with how weewx addresses upgrades). >>> Everything is stable, so I'm moving onward to the next steps. >>> >>> >>> >>> I've taken the route to implement WeeWX SteelSeries Weather Gauges >>> extension (not real-time yet) and WeeWX-WD v2.1.2 -- everything installed >>> and I can see the SteelGauge info on my remotely hosted website, as well as >>> see the clientraw.text and other key files for Saratoga Templates on the >>> remotely hosted server. >>> >>> >>> >>> However, even though I have the archive_interval in weewx.conf set to >>> 120 (seconds) the FTP of the files generated by weewx is occurring every >>> 300 seconds (5 minutes) -- which I confirmed from reviewing the log >>> files. How can I get the FTP to execute every 120 seconds? >>> >>> >>> >>> FYI, weewx.conf for Standard Archive is >>> >>> >>> >>> >>> >>> >>> >>> >>> *[StdArchive] archive_interval = 120 record_generation = >>> hardware loop_hilo = True data_binding = wx_binding* >>> >>> >>> >>> Should I change the record_generation to software? Noting the Rainwise >>> IP-100 is inserting records into the MySQL database tables every minute -- >>> which is what I want. >>> >>> >>> >>> Thanks in Advance for your guidance! >>> >>> >>> >>> -- Tim >>> >>> >>> >>> On Saturday, January 30, 2021 at 8:35:31 PM UTC-8 tim lambert wrote: >>> >>> Thanks Gary for the insight and guidance. >>> >>> >>> >>> >>> >>> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for >>> Windows 10 >>> >>> >>> >>> *From: *gjr80 >>> *Sent: *Saturday, January 30, 2021 7:58 PM >>> *To: *weewx-user >>> *Subject: *[weewx-user] Re: WEEWX-WD and Weewx 4.2 to 4.4 >>> >>> >>> >>> Hi, >>> >>> >>> >>> I wouldn't be using WeeWX-WD from bitbucket, it has not been touched for >>> some years now. WeeWX-WD on github >>> <https://github.com/gjr80/weewx-weewx-wd> is more up to date, the >>> current version is v2.1.2 from November 2020. It should have no problems >>> running on WeeWX v4.x under python 2 or python3. The SteelSeries >>> Weather Gauges <https://github.com/mcrossley/SteelSeries-Weather-Gauges> >>> have directly supported WeeWX of v2.5.0 (of the SteelSeries Weather Gauges) >>> or for about six years, and there is now a WeeWX SteelSeries Gauges >>> extension <https://github.com/weewx/weewx/wiki/steelseries> that >>> installs the necessary skin to support the SteelSeries Weather Gauges (this >>> is the preferred means of providing the SteelSeries Weather Gauges, >>> WeeWX-WD did support the SteelSeries Weather Gauges via the WD >>> customclientraw.txt but since the advent of direct WeeWX support and >>> the WeeWX extension the SteelSeries Weather Gauges support has been >>> deprecated in WeeWX-WD). You should find the WeeWX extension will work >>> under WeeWX v4.x and python2 or python3. >>> >>> >>> >>> I would suggest that you upgrade to WeeWX v4.4.0 and get WeeWX running >>> to your satisfaction before installing either WeeWX-WD v2.1.2 or the WeeWX >>> SteelSeries Weather Gauges extension (in whatever order suits). Once you >>> have the SteelSeries Weather Gauges extension running OK you can, if you >>> want, look at installing the Realtime gauge-data extension >>> <https://github.com/gjr80/weewx-realtime_gauge-data> which will allow >>> the gauges to update on every loop packet rather than on each report cycle >>> (how often they will update will depend on how often your station emits >>> loop packets, for your Rainwise this should be up to every couple of >>> seconds). >>> >>> >>> >>> Gary >>> >>> >>> >>> On Sunday, 31 January 2021 at 13:15:24 UTC+10 tim lambert wrote: >>> >>> Gang, >>> >>> >>> >>> Currently running weewx 4.2 (will be upgrading in the next few days to >>> 4.4) on a Raspberry Pi 4 (32-bit OS) with a Rainwise MK-III and IP-100 >>> providing station data -- everything has been working properly for several >>> weeks and is populating a remote MySQL DB. >>> >>> >>> >>> Once I upgrade weewx, I'd like to incorporate using Steel Gauges and >>> Saratoga Templates, hence my query: Can I utilize the WEEWX-WD as outlined >>> BitBucket >>> Weewx-WD <https://bitbucket.org/ozgreg/weewx-wd/wiki/Users%20Guide>? >>> >>> >>> >>> The Steel Gauges and Saratoga Templates will be hosted on a remote, >>> public accessible server -- so I'll be sending the clientraw and testtag >>> files via FTP. >>> >>> >>> >>> Thanks in Advance! >>> >>> >>> >>> - Tim >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "weewx-user" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/weewx-user/gvReyZeqOJQ/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/6942d0aa-94bf-46e9-bb56-8232a425f95fn%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/weewx-user/6942d0aa-94bf-46e9-bb56-8232a425f95fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/e8c94cec-3d75-4ac1-93d3-19cfcae8a1b6n%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/weewx-user/e8c94cec-3d75-4ac1-93d3-19cfcae8a1b6n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "weewx-user" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/weewx-user/gvReyZeqOJQ/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/CAPq0zEC0JotC0Rrp0NirvcUx8%3DvL6kzY9W7%3D1SbdU75_ke1i%3DQ%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/weewx-user/CAPq0zEC0JotC0Rrp0NirvcUx8%3DvL6kzY9W7%3D1SbdU75_ke1i%3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/ECA9484A-7708-449E-B0C1-DDB22A0F9DEA%40comcast.net >>> >>> <https://groups.google.com/d/msgid/weewx-user/ECA9484A-7708-449E-B0C1-DDB22A0F9DEA%40comcast.net?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "weewx-user" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/weewx-user/gvReyZeqOJQ/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/CAPq0zEAK_512pG%3DDPbghbVCnJSJQjk2Q9uEHW12g4WEZkP4CTw%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/weewx-user/CAPq0zEAK_512pG%3DDPbghbVCnJSJQjk2Q9uEHW12g4WEZkP4CTw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/601d8a3f.1c69fb81.c2a39.613cSMTPIN_ADDED_MISSING%40gmr-mx.google.com >>> >>> <https://groups.google.com/d/msgid/weewx-user/601d8a3f.1c69fb81.c2a39.613cSMTPIN_ADDED_MISSING%40gmr-mx.google.com?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "weewx-user" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/weewx-user/gvReyZeqOJQ/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/CAPq0zEB9Fcu3tLZpGVFqhTqoQjXYzLgMBP-R5vjRqBx-jyXVQw%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/weewx-user/CAPq0zEB9Fcu3tLZpGVFqhTqoQjXYzLgMBP-R5vjRqBx-jyXVQw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/7a3c3f02-37e0-4ba1-9a45-d45797eee986n%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/weewx-user/7a3c3f02-37e0-4ba1-9a45-d45797eee986n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "weewx-user" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/weewx-user/gvReyZeqOJQ/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/CAPq0zEAKnx3HS_8g-OBz%2BoAkNu9ky-Z6e1xp9DD8HNFGwmxeZg%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/weewx-user/CAPq0zEAKnx3HS_8g-OBz%2BoAkNu9ky-Z6e1xp9DD8HNFGwmxeZg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/601db9f5.1c69fb81.d7df7.95d2SMTPIN_ADDED_MISSING%40gmr-mx.google.com >>> >>> <https://groups.google.com/d/msgid/weewx-user/601db9f5.1c69fb81.d7df7.95d2SMTPIN_ADDED_MISSING%40gmr-mx.google.com?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "weewx-user" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/weewx-user/gvReyZeqOJQ/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/3b181fa0-029d-4c57-8649-4a49b3413a7fn%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/weewx-user/3b181fa0-029d-4c57-8649-4a49b3413a7fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> >>> >> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/f2c90dc3-f29b-4d2d-bcad-61d3f2093f74n%40googlegroups.com.
