1. You are getting memory leaks in 4.0.0 running both Python 2 and Python 3?
2. The biggest architectural differences between v3.9 and v4.0 is in how
derived types are handled (V4 uses the new extensible type system) and in
how strings are handled (v4 uses unicode), The former is pure Python. The
latter is not, but I've got to assume that the Python Software Foundation
has thoroughly debugged that code.
3. Cheetah uses an underlying C library ("NameMapper"). It's possible they
screwed up something in their Python 3 port.
That's all I can think of off the top of my head.
-tk
On Sun, Mar 29, 2020 at 5:11 PM Lucas Heijst <ljm.hei...@gmail.com> wrote:
> Tom,
>
> I I forgot to say these same set of programs and webcam cronjob were
> running since a year now with weewx 3.9.1, raspian stretch and python2.
> Now they are running weewx 4.00b16-17, raspbian stretch and python2 or
> python3 with the memory problems mentioned.
>
> Luc
>
> On Sunday, 29 March 2020 21:03:03 UTC-3, Lucas Heijst wrote:
>>
>> Tom,
>>
>> Due to memory grow my system crashed after 9-10 days. So I wrote here
>> that I would implement a cron job to restart my system after 5 days.
>> Vice Skahan convinced me to trace down the cause of the memory leaks and
>> that I did.
>>
>> No more, no less.
>>
>> Luc
>>
>> On Sunday, 29 March 2020 20:54:56 UTC-3, Tom Keffer wrote:
>>>
>>> I'm sorry, Luc, not trying to be difficult, but what is your question? I
>>> can't possibly debug your memory leak, especially with the complicated set
>>> of drivers and skins you are running.
>>>
>>> WeeWX itself does not have any memory leaks that I'm aware of. It is
>>> 100% Python, and Python uses garbage collection. If there are circular
>>> references, it can take a while for the interpreter to find unused and
>>> recycle memory, but WeeWX does not use any circular references.
>>>
>>> Here's an example <http://www.threefools.org/weewx/status/index.html>
>>> of a station that has been running continuously for well over a year.
>>> Present memory use is VIRT=75060 RES=37736 SHR=7144. So, very long runtimes
>>> are possible without leaks.
>>>
>>> Net-net, if there are leaks, I am pretty confident they are in the
>>> underlying C libraries.
>>>
>>> -tk
>>>
>>> On Sun, Mar 29, 2020 at 3:50 PM Lucas Heijst <ljm....@gmail.com> wrote:
>>>
>>>> Tom,
>>>>
>>>> I have experienced two types of memory leak.
>>>> 1. In cronjobs when defining variables without unset them afterwards.
>>>> 2. With weewx reports (mben with 84+ generated graphs each run).
>>>>
>>>> Luc
>>>>
>>>> On Sunday, 29 March 2020 19:10:52 UTC-3, Tom Keffer wrote:
>>>>>
>>>>> By the way, in my experience, these leaks are caused by underlying "C"
>>>>> libraries or, occasionally, by failure of PIL to release graphics resource
>>>>> (such as font handles).
>>>>>
>>>>> On Sun, Mar 29, 2020 at 3:09 PM Thomas Keffer <tke...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I'm not following. I thought you said you could get rid of the leak
>>>>>> by doing the unsets. If so, is there still a problem?
>>>>>>
>>>>>>
>>>>>> On Sun, Mar 29, 2020 at 8:41 AM Lucas Heijst <ljm....@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Tom,
>>>>>>>
>>>>>>> Currently running weewx 4.0.0b17 and analysing the memory leak in my
>>>>>>> pi31 system.
>>>>>>> On the system runs two weewx instances (mben and tfrc) and a
>>>>>>> 5-minute webcam cronjob.
>>>>>>>
>>>>>>> The memory leak of my webcam cronjob task was caused by creating
>>>>>>> variables like:
>>>>>>> WEBCAM_ID=3
>>>>>>> CAMERA=picamera$WEBCAM_ID
>>>>>>> DATETIME=$(date +"%Y-%m-%d %H:%M:%S")
>>>>>>> EPOCH=$(date +"%s")
>>>>>>> ...
>>>>>>>
>>>>>>> The used memory is freed by:
>>>>>>> unset WEBCAM_ID
>>>>>>> unset CAMERA
>>>>>>> unset DATETIME
>>>>>>> unset EPOCH
>>>>>>> ...
>>>>>>>
>>>>>>>
>>>>>>> The memory leak of the weewx instances mben and tfrc are caused by
>>>>>>> the weewx reporting tasks, see data below:
>>>>>>>
>>>>>>> -----
>>>>>>> used time
>>>>>>> mem
>>>>>>> ----- -----
>>>>>>> 73348 10:30
>>>>>>> 95504 11:07
>>>>>>> 48576 11:38 no webcam, no mben, no tfrc (no increase of memory)
>>>>>>> 56136 11:41 mben without modbus read and modbus service
>>>>>>> 57384 11:43
>>>>>>> 57876 11:44
>>>>>>> 58360 11:45
>>>>>>> 64836 11:47 used memory increased with 6476 due to mben31 report
>>>>>>> 64572 11:48
>>>>>>> 64560 11:49
>>>>>>> 64548 11:50
>>>>>>> 66240 11:51 used memory increased with 1692 due to mben31 report
>>>>>>> 55808 11:53 mben without mben31 report
>>>>>>> 56092 11:54
>>>>>>> 55828 11:55
>>>>>>> 55808 11:56
>>>>>>> 56000 11:57
>>>>>>> 56000 11:58
>>>>>>> 56516 11:59
>>>>>>> 56772 12:00
>>>>>>> 56164 12:02 mben now with modbus service (fully functional; without
>>>>>>> reporting)
>>>>>>> 56436 12:03
>>>>>>> 55920 12:04
>>>>>>> 76056 12:09 mben without mben31 report; tfrc started without tfrc31
>>>>>>> report
>>>>>>> 76388 12:10
>>>>>>> 77084 12:11
>>>>>>> 77248 12:12
>>>>>>> 77432 12:13
>>>>>>> 77312 12:14
>>>>>>> 76908 12:15
>>>>>>> 76976 12:16
>>>>>>> 77268 12:21
>>>>>>> 77248 12:24 hardly any increase of used memory the last 15 minutes
>>>>>>> -----
>>>>>>>
>>>>>>> When all reports are disabled, the memory leak on my system is
>>>>>>> practically zero.
>>>>>>>
>>>>>>> Attached the skin settings of mben31.
>>>>>>>
>>>>>>> Any clue how reduce this memory leak?
>>>>>>>
>>>>>>> Luc
>>>>>>>
>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "weewx-development" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to weewx-de...@googlegroups.com.
>>>>>>> To view this discussion on the web visit
>>>>>>> https://groups.google.com/d/msgid/weewx-development/d91724fe-da46-4b4e-9913-59f55aa91657%40googlegroups.com
>>>>>>> <https://groups.google.com/d/msgid/weewx-development/d91724fe-da46-4b4e-9913-59f55aa91657%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>>> .
>>>>>>>
>>>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "weewx-development" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to weewx-de...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/weewx-development/b5c12aeb-c8e1-489e-b457-04d2f8783ca0%40googlegroups.com
>>>> <https://groups.google.com/d/msgid/weewx-development/b5c12aeb-c8e1-489e-b457-04d2f8783ca0%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>> --
> You received this message because you are subscribed to the Google Groups
> "weewx-development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-development+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-development/f0a096da-16c4-48cc-86e6-13e9f18fba96%40googlegroups.com
> <https://groups.google.com/d/msgid/weewx-development/f0a096da-16c4-48cc-86e6-13e9f18fba96%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-development+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-development/CAPq0zECokAAk53%2BUtHjXactNNsLdJZ8_YzCEcK8ngurCa5B8WA%40mail.gmail.com.
