Thanks, will do but later today. Regards,
Garry Lockyer C: +1.250.689.0686 E: ga...@lockyer.ca > On Jan 16, 2021, at 11:09, bell...@gmail.com <bellri...@gmail.com> wrote: > > > Garry, > If you want me to run something, just make it available and give me some > instructions. > rich >> On Saturday, 16 January 2021 at 12:25:33 UTC-5 garrya...@gmail.com wrote: >> I've added the mem extension. My test station, with a large number of feeds >> selected, is at: https://lockyer.ca/weather/OsoyoosLakeNorthEast >> >> mem graphs are at: https://lockyer.ca/weather/OsoyoosLakeNorthEast/mem >> >> Regards, >> >> Garry >> >>> On Saturday, January 16, 2021 at 8:42:09 AM UTC-8 garrya...@gmail.com wrote: >>> Brief Update: January 16, 2021: >>> >>> I've pretty much finished moving my extension to a service. >>> >>> I started WeeWX at about midnight and at about 8:20 AM, memory usage has >>> grown to over 941MB! Allocated blocks was at 4,245,722. >>> >>> So growing memory usage problem exists. >>> >>> Bill indicated above he ran my test code on Ubuntu so I set up a Ubuntu >>> Raspberry Pi Desktop system: >>> >>> RPi 4 - 6 GB >>> Ubuntu Raspberry Pi OS 20.10 >>> Python3 - 3.8.6 >>> WeeWX 4.3 >>> Belchertown 1.2 >>> >>> Later today I intend to add the mem extension. Will report back soonest! >>> I will also run things on Raspberrry Pi OS (but I have to build a new one!). >>> >>> Regards, >>> >>> Garry >>> PS: I *think* I'm going to continue all development on Ubuntu as it seems >>> more current - it has a later linux kernel and more current Python - but >>> I'm not sure it's otherwise better than Raspberry Pi OS. Both OS's have >>> wireless mouse lag problems, fixed somewhat by changing 'mousepoll' >>> setting. Installation of Ubuntu was easy and I managed to get it to boot >>> from a USB SSD. In the first 8 hours or so, Ubuntu froze a couple of >>> times, so now I keep an ssh session open so that I can reboot. >>> >>>> On Saturday, January 9, 2021 at 8:02:51 PM UTC-8 garrya...@gmail.com wrote: >>>> My experience is that when I recreate an include file for Belchertown on >>>> each archive cycle, as an extension to Belchertown or as a service, memory >>>> usage increases with each cycle and eventually errors start. I’ve seen >>>> the error I reported in this string and unresponsive systems. So, yes, I >>>> think there’s a problem. >>>> >>>> The memory test service I provided starts out at about 45 MB and increases >>>> at about 3 MB per cycle. I’ve seen WeeWx memory usage grow to over 1 GB! >>>> >>>> I’m working on a new version of my extension, as a service, and will >>>> report back on its memory usage, hopefully within only a couple of days. >>>> >>>> Thanks for all your help! >>>> >>>> >>>> Regards, >>>> >>>> Garry Lockyer >>>> C: +1.250.689.0686 >>>> E: ga...@lockyer.ca >>>> >>>> >>>>>> On Jan 9, 2021, at 19:11, vince <vince...@gmail.com> wrote: >>>>>> >>>>> If your memory usage is stable, is there a problem ? >>>> >>>>> It's pretty typical after a weewx startup for the usage to go up a bit >>>>> then level off nicely. >>>>> >>>>>> On Saturday, January 9, 2021 at 4:41:51 PM UTC-8 bell...@gmail.com wrote: >>>>>> Garry, >>>>>> I ran your test extension for a bit in my Ubuntu VM. Doesn't appear to >>>>>> be leaking. >>>>>> 16:42:24 {'mem_size': 338.53515625, 'mem_rss': 32.953125, 'mem_share': >>>>>> 11.28125} >>>>>> 16:47:38 {'mem_size': 423.80859375, 'mem_rss': 47.75, 'mem_share': >>>>>> 11.796875} >>>>>> 16:52:53 {'mem_size': 424.4140625, 'mem_rss': 48.2265625, 'mem_share': >>>>>> 11.796875} >>>>>> 16:57:24 {'mem_size': 426.46875, 'mem_rss': 50.19921875, 'mem_share': >>>>>> 11.796875} >>>>>> 17:02:32 {'mem_size': 426.46875, 'mem_rss': 50.21875, 'mem_share': >>>>>> 11.796875} >>>>>> 17:07:20 {'mem_size': 427.1796875, 'mem_rss': 50.9921875, 'mem_share': >>>>>> 11.796875} >>>>>> 17:12:27 {'mem_size': 427.1796875, 'mem_rss': 50.9921875, 'mem_share': >>>>>> 11.796875} >>>>>> 17:17:26 {'mem_size': 427.1796875, 'mem_rss': 50.9921875, 'mem_share': >>>>>> 11.796875} >>>>>> 17:22:24 {'mem_size': 427.1796875, 'mem_rss': 50.9921875, 'mem_share': >>>>>> 11.796875} >>>>>> 17:27:19 {'mem_size': 424.6640625, 'mem_rss': 48.71875, 'mem_share': >>>>>> 11.796875} >>>>>> 17:32:27 {'mem_size': 424.6640625, 'mem_rss': 48.71875, 'mem_share': >>>>>> 11.796875} >>>>>> 17:37:17 {'mem_size': 424.5859375, 'mem_rss': 48.7109375, 'mem_share': >>>>>> 11.796875} >>>>>> 17:42:21 {'mem_size': 424.58203125, 'mem_rss': 48.70703125, >>>>>> 'mem_share': 11.796875} >>>>>> 17:47:24 {'mem_size': 424.984375, 'mem_rss': 49.140625, 'mem_share': >>>>>> 11.796875} >>>>>> 17:52:21 {'mem_size': 424.984375, 'mem_rss': 49.140625, 'mem_share': >>>>>> 11.796875} >>>>>> 17:57:15 {'mem_size': 425.05078125, 'mem_rss': 49.26953125, >>>>>> 'mem_share': 11.796875} >>>>>> 18:02:11 {'mem_size': 425.05078125, 'mem_rss': 49.26953125, >>>>>> 'mem_share': 11.796875} >>>>>> 18:07:13 {'mem_size': 425.1171875, 'mem_rss': 49.3359375, 'mem_share': >>>>>> 11.796875} >>>>>> 18:12:14 {'mem_size': 425.1171875, 'mem_rss': 49.3359375, 'mem_share': >>>>>> 11.796875} >>>>>> 18:17:12 {'mem_size': 425.21875, 'mem_rss': 49.4375, 'mem_share': >>>>>> 11.796875} >>>>>> 18:22:10 {'mem_size': 425.09765625, 'mem_rss': 49.31640625, >>>>>> 'mem_share': 11.796875} >>>>>> 18:27:13 {'mem_size': 425.0390625, 'mem_rss': 49.2578125, 'mem_share': >>>>>> 11.796875} >>>>>> 18:32:15 {'mem_size': 425.0390625, 'mem_rss': 49.2578125, 'mem_share': >>>>>> 11.796875} >>>>>> 18:37:15 {'mem_size': 425.03125, 'mem_rss': 49.25, 'mem_share': >>>>>> 11.796875} >>>>>> 18:42:24 {'mem_size': 425.01953125, 'mem_rss': 49.2578125, 'mem_share': >>>>>> 11.796875} >>>>>> 18:47:14 {'mem_size': 425.0078125, 'mem_rss': 49.24609375, 'mem_share': >>>>>> 11.796875} >>>>>> 18:52:13 {'mem_size': 425.0078125, 'mem_rss': 49.24609375, 'mem_share': >>>>>> 11.796875} >>>>>> 18:57:15 {'mem_size': 425.61328125, 'mem_rss': 49.8515625, 'mem_share': >>>>>> 11.796875} >>>>>> 19:02:19 {'mem_size': 425.0078125, 'mem_rss': 49.2734375, 'mem_share': >>>>>> 11.796875} >>>>>> >>>>>> When I have some time, I'll install on my test pi along with the >>>>>> Belchertown skin and Vince's mem extension and see if I see anything. >>>>>> Yes, I enjoy a good memory 'puzzle'. >>>>>> rich >>>>>>> On Saturday, 9 January 2021 at 16:52:08 UTC-5 garrya...@gmail.com wrote: >>>>>>> Many, many thanks! >>>>>>> >>>>>>> Before I wrote the service to investigate the problem, I wrote a >>>>>>> standalone program to eliminate all WeeWX components. It did NOT >>>>>>> exhibit the problem. >>>>>>> >>>>>>> Problem only seems to show up under WeeWX/Belchertown *and* if the >>>>>>> include file changes / is recreated. >>>>>>> >>>>>>> Anyway, I really appreciate you looking into my problem. I’m actively >>>>>>> working on a skin-independent service to read many RSS/Atom feeds and >>>>>>> will utilize your suggestions above. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Garry Lockyer >>>>>>> C: +1.250.689.0686 >>>>>>> E: ga...@lockyer.ca >>>>>>> >>>>>>> >>>>>>>>> On Jan 9, 2021, at 12:39, bell...@gmail.com <bell...@gmail.com> wrote: >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>>> Garry, >>>>>>>> I took your WxFeedsMemory.py and made some modifications. >>>>>>>> First, I changed the logging to simple print calls. this was just a >>>>>>>> convenience for me to easily see what was going on. >>>>>>>> Second, I stole some code from Vince’s mem extension, >>>>>>>> https://github.com/vinceskahan/vds-weewx-v3-mem-extension. I did this >>>>>>>> because I am used to the data it returns. (Note, I highly recommend >>>>>>>> this when debugging memory.) >>>>>>>> Third, I added code so I could call your extension repeatedly in a >>>>>>>> loop. >>>>>>>> >>>>>>>> When I called new_archive_record 100 times there was a slight increase >>>>>>>> in memory, but nothing alarming. But it was slow, approximately 2 >>>>>>>> minutes a call. >>>>>>>> >>>>>>>> So, I commented out the flush and fsync calls and called >>>>>>>> new_archive_record 10,000 times. The slight increase seemed to stop >>>>>>>> around 6,000 calls. >>>>>>>> >>>>>>>> I’m wondering if the amount of time it takes to run is causing >>>>>>>> problems when run as a service. I plan to uncomment the code and run >>>>>>>> it as an extension. I’ll let you know what I see. >>>>>>>> rich >>>>>>>> >>>>>>>> >>>>>>>>> On Thursday, 7 January 2021 at 13:00:40 UTC-5 garrya...@gmail.com >>>>>>>>> wrote: >>>>>>>>> After many hours of troubleshooting and testing, I think I have an >>>>>>>>> idea what's happening. >>>>>>>>> >>>>>>>>> Background: I want to use weewx with the Belchertown skin and my >>>>>>>>> extension that reads numerous RSS/Atom feeds on a Pi Zero, with >>>>>>>>> either a Davis Pro V2 or Weatherflow weather station. I will >>>>>>>>> eventually have four weather stations in my area. I thought I had >>>>>>>>> things working reasonably well so deployed one station. It stopped >>>>>>>>> working after about 3 days. I can't access that system remotely so I >>>>>>>>> built a very similar system (the next one to deploy) and it falso >>>>>>>>> ailed due to lack of memory. Weewx memory usage on my development >>>>>>>>> system also increases over time - it can grow to over 1 GB in a few >>>>>>>>> hours! >>>>>>>>> >>>>>>>>> My extension was an extension to Belchetown based on the METAR >>>>>>>>> extension. The extension created Belchertown index hook include >>>>>>>>> files each archive cycle. While researching this problem I re-read >>>>>>>>> the weewx customization guide and noted that extensions should NOT be >>>>>>>>> dependent on other extensions so I decided to re-write my extension >>>>>>>>> as a service (which looks like a much cleaner solution) without any >>>>>>>>> dependencies on Belchertown (other than include file names). All >>>>>>>>> I've done so far is create a service (WxFeedsMemoryTest.py) to test >>>>>>>>> weewx/Belchertown memory consumption. >>>>>>>>> >>>>>>>>> The Problem: weewx/Bechertown memory usage increases over time. It >>>>>>>>> starts out at about 45 MB and grows at about 3MB per archive >>>>>>>>> period/cycle (using my test case). A 512 MB Pi will exhaust memory >>>>>>>>> within a few days. >>>>>>>>> >>>>>>>>> It appears that the problem is associated with the creation of the >>>>>>>>> Belchertown include files while weewx/Belchertown is running: >>>>>>>>> >>>>>>>>> - if the include file is 'static' as in not (re)created while >>>>>>>>> weewx/Belchertown is running, memory usage is static - it does not >>>>>>>>> grow beyond about 50 MB. >>>>>>>>> >>>>>>>>> - if the include file is 'dynamic' as in (re)created while >>>>>>>>> weewx/Belchetown is running, memory usage increases. >>>>>>>>> - if the include file is created once, and becomes 'static', memory >>>>>>>>> usage increases and then stabilizes. >>>>>>>>> - if the include file is recreated continuously (such as on each >>>>>>>>> archive cycle), memory usage increases each cycle. >>>>>>>>> >>>>>>>>> It does not appear to matter if the include file is created directly, >>>>>>>>> or created as a temporary file and then copied or renamed. >>>>>>>>> >>>>>>>>> The attached service (WxFeedsMemoryTest.py) can be used to >>>>>>>>> demonstrate the problem. Please see installation and use >>>>>>>>> instructions within the WxFeedsMemoryTest.py. >>>>>>>>> >>>>>>>>> I'm going to continue to work on moving my extension from "an >>>>>>>>> extension to an extension" to a service in the hope that this memory >>>>>>>>> problem can be resolved. >>>>>>>>> >>>>>>>>> With apologies in advance if I'm doing something to cause the >>>>>>>>> problem, please review, advise and let me know what I can do to avoid >>>>>>>>> the problem. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Garry >>>>>>>>>> On Thursday, December 31, 2020 at 7:05:15 PM UTC-8 >>>>>>>>>> garrya...@gmail.com wrote: >>>>>>>>>> Got MemoryError after about 9 hours after restart. Have removed >>>>>>>>>> cmon by commenting out any mention of cmon in weewx.conf and >>>>>>>>>> restarted. >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> >>>>>>>>>> Garry Lockyer >>>>>>>>>> Former DEC Product Support Engineer :^) >>>>>>>>>> Kepner-Tregoe Trained :^)) >>>>>>>>>> C: +1.250.689.0686 >>>>>>>>>> E: ga...@lockyer.ca >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>>> On Dec 31, 2020, at 11:44, vince <vince...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Thursday, December 31, 2020 at 11:39:39 AM UTC-8 >>>>>>>>>>> garrya...@gmail.com wrote: >>>>>>>>>>>> Re: editing the Belchertown skin, nope haven’t touched it, other >>>>>>>>>>>> than interfacing with it via the include files (as generated by my >>>>>>>>>>>> BelchertownWxFeeds extension). When all the endpoints (for >>>>>>>>>>>> testing) are enabled index.html is about 1.8MB, so perhaps that’s >>>>>>>>>>>> causing the problem. I can easily reduce / eliminate endpoints >>>>>>>>>>>> and prefer to do that before eliminating the Belchertown skin. >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> There it is. You touched it :-) >>>>>>>>>>> >>>>>>>>>>> Usual debugging rules apply. Reset it to a baseline unmodified >>>>>>>>>>> config. Add in changes one-by-one. If it goes sideways, revert to >>>>>>>>>>> the last known good and reverify that it stays good. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>>>> Groups "weewx-user" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>>> send an email to weewx-user+...@googlegroups.com. >>>>>>>>>> >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/57a5bfc7-d0b4-4419-b178-6342564642edn%40googlegroups.com. >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "weewx-user" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>>>> an email to weewx-user+...@googlegroups.com. >>>>>>> >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/weewx-user/d5e41e91-5dab-41bc-bc34-682826ba4753n%40googlegroups.com. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send an >>>>> email to weewx-user+...@googlegroups.com. >>>> >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/weewx-user/26d2e298-1f5a-4d54-8cc8-89acaf65dab3n%40googlegroups.com. > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to weewx-user+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/0f3d3ae7-ad42-4ebb-a395-81ddc2887496n%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/2FDA9B0E-B731-4494-B9C7-9017F634F6FF%40gmail.com.
