Gary, The programs modbusenergy.py, tfrc.py and cmon.py are modified to use module logging instead of syslog. Everything still works OK.
Program modbusenergy has a memory leak of 80 KB per 24 h (nothing new), so the memory will be full after 7.5 days. It looks like the program starts and stops a program twice per second, because each time I perform 'ps aux' the count is higher (2 counts per second). Currently the counter has reach a maximum of 32316 and does't increase anymore. I have too little knowledge of linux and python to debug the memory leak, so as a work-around my system is rebooted each 5th day. Luc On Thursday, 26 March 2020 14:24:27 UTC-3, Lucas Heijst wrote: > > Thanks Gary, > > In no means I wanted to be hars, sorry about that. I am Dutch and we > usualy say things with less diplomacy than others. > Currently I am gathering the information you asked for. > > The good news is that the following combination seems to work OK: > ----- > RPI 3B+ > weewx version 4.0.0b16 > Python 2.7.13 > Platform Linux-4.19.66-v7+-armv7l-with-debian-9.11 (latest version of > raspbian jessie) > ----- > The weewx programs modbusenergy.py, tfrc.py and cmon.py (used with weewx > 3.9.2) were not modified so far. > The difference between version 3.9.2 and 4.0.0.b16 for the data in > /var/log/syslog is the debug logging which will not be suppressed with > setting ‘debug = 0’ for the program messages generated with: > def logmsg(dst, msg): > syslog.syslog(dst, 'ModbusEnergy: %s' % msg) > I will let this configuration run for 24 hours and then modify the three > user.py files to use logging: > log = logging.getLogger(__name__) instead of syslog. > I will keep you informed. > > Luc > > On Thursday, 26 March 2020 04:47:37 UTC-3, gjr80 wrote: >> >> On Thursday, 26 March 2020 13:21:55 UTC+10, Lucas Heijst wrote: >> >>> I think the fact that weewx can't keep up in time to handle the 1-minute >>> archive data is the reason that the memory is filled. >>> >> >> I think you are being a little harsh here Luc. From what i can read of >> the log extracts you are running a rather complex setup (two instances of >> WeeWX and at least three databases). I can think of a number of scenarios >> that could cause the symptoms you are seeing and none involve WeeWX not >> 'keeping up'. >> >> >>> Each archive record contains 90 real values (doubles) and these values >>> are stored in memory (I suppose) as long as they are not written to the >>> database. >>> >> One hour before the low-memory crash weewx was 90 1-minute archive >>> records behind. >>> >> >> Might help to have a little more info if you want any meaningful advice >> other than maybe this or maybe that. >> >> Are you running with software archive record generation? If so then WeeWX >> does not store any archive records (other than at most two, the last one >> that is being processed and the current one that is being accumulated). >> WeeWX accumulates loop packets and as soon as WeeWX receives a loop packet >> that belongs in the next archive period WeeWX synthesises an archive record >> and emits a NEW_ARCHIVE_RECORD event which causes that archive record to be >> archived and otherwise processed by the WeeWX services. The loop packet >> from the next archive period (and subsequent loop packets) are accumulated >> to create the next archive record. And the process repeats. So on a system >> with a driver that normally emits loop packets every 4 seconds, but for >> some reason these loop packets are being blocked within the driver and >> released much less frequently, WeeWX will only emit an archive record once >> it has received a loop packet timestamped in the next archive period. So >> what you can see happening is archive records being emitted and saved to >> database but at intervals much longer than the archive interval, in many >> cases the interval between these archive records being emitted increases as >> more and more loop packets are delayed. Ultimately an out of memory error >> can occur, if things haven't ground to a halt beforehand. >> >> How about providing some config details? A debug log from startup would >> show almost everything needed to gicve us a more complete picture of your >> setup. What about your driver, does it interrogate hardware in its own >> thread or in the WeeWX thread? If it is not in its own thread it is >> possible that it is blocking and causing delays. What happens when you run >> WeeWX directly, do you see loop packets appearing, does the difference in >> successive loop packet timestamps align with the frequency they appear on >> the console? (ie if the loop packets are timestamped every four seconds do >> they appear on the console every fours seconds or every 15 seconds, 30 >> seconds, 60 seconds?) If there is a discrepancy that is a sure sign that >> something is blocking. How about posting a copy of the driver? >> >> Hard to say a great deal without more logs, more config details and more >> driver details. >> >> Gary >> > -- You received this message because you are subscribed to the Google Groups "weewx-development" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-development+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/03c77d5b-a2eb-488d-89b4-65972544de5b%40googlegroups.com.
