Hi Tom Thank you for the explanation. WeeWx system would run on again due to restart if I understand right. Because I don't see any additional messages in the log I cann't catch this (might make sense to add some in the future release?), anyway when PWS come back online data are not stored in DB. Have I any chance to get/trace status where the WeeWx is? My guess is that DB engine is maybe stoped. I made a test and restart MySQL when WeeWx run jobs even PWS is online and I got this in log and same situation as PWS is offline.
MySQl log: *171007 6:35:13 [Note] /usr/sbin/mysqld: Normal shutdown171007 6:35:13 [Note] Event Scheduler: Purging the queue. 0 events171007 6:35:15 [Warning] /usr/sbin/mysqld: Forcing close of thread 21567 user: 'mail'171007 6:35:15 [Warning] /usr/sbin/mysqld: Forcing close of thread 18190 user: 'weewx'171007 6:35:15 [Warning] /usr/sbin/mysqld: Forcing close of thread 18189 user: 'weewx'171007 6:35:15 InnoDB: Starting shutdown...171007 6:35:16 InnoDB: Waiting for 113 pages to be flushed171007 6:35:17 InnoDB: Shutdown completed; log sequence number 17285249150171007 6:35:17 [Note] /usr/sbin/mysqld: Shutdown complete171007 6:35:19 [Note] Plugin 'FEDERATED' is disabled.171007 6:35:19 InnoDB: The InnoDB memory heap is disabled171007 6:35:19 InnoDB: Mutexes and rw_locks use GCC atomic builtins171007 6:35:19 InnoDB: Compressed tables use zlib 1.2.8171007 6:35:19 InnoDB: Using Linux native AIO171007 6:35:19 InnoDB: Initializing buffer pool, size = 512.0M171007 6:35:19 InnoDB: Completed initialization of buffer pool171007 6:35:19 InnoDB: highest supported file format is Barracuda.171007 6:35:19 InnoDB: Waiting for the background threads to start171007 6:35:20 InnoDB: 5.5.57 started; log sequence number 17285249150171007 6:35:20 [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306171007 6:35:20 [Note] - '127.0.0.1' resolves to '127.0.0.1';171007 6:35:20 [Note] Server socket created on IP: '127.0.0.1'.171007 6:35:20 [Note] Event Scheduler: Loaded 0 events171007 6:35:20 [Note] /usr/sbin/mysqld: ready for connections.Version: '5.5.57-0+deb8u1-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Debian)* WeeWx log: *Oct 7 06:35:13 ferda weewx[27420]: cheetahgenerator: Generated 14 files for report StandardReport in 1.49 secondsOct 7 06:35:13 ferda weewx[27420]: reportengine: Caught unrecoverable exception in generator weewx.imagegenerator.ImageGeneratorOct 7 06:35:13 ferda weewx[27420]: **** (2006, 'MySQL server has gone away')Oct 7 06:35:13 ferda weewx[27420]: **** Traceback (most recent call last):Oct 7 06:35:13 ferda weewx[27420]: **** File "/usr/share/weewx/weewx/reportengine.py", line 239, in runOct 7 06:35:13 ferda weewx[27420]: **** obj.start()Oct 7 06:35:13 ferda weewx[27420]: **** File "/usr/share/weewx/weewx/reportengine.py", line 273, in startOct 7 06:35:13 ferda weewx[27420]: **** self.run()Oct 7 06:35:13 ferda weewx[27420]: **** File "/usr/share/weewx/weewx/imagegenerator.py", line 31, in runOct 7 06:35:13 ferda weewx[27420]: **** self.genImages(self.gen_ts)Oct 7 06:35:13 ferda weewx[27420]: **** File "/usr/share/weewx/weewx/imagegenerator.py", line 141, in genImagesOct 7 06:35:13 ferda weewx[27420]: **** aggregate_interval=aggregate_interval)Oct 7 06:35:13 ferda weewx[27420]: **** File "/usr/share/weewx/weewx/manager.py", line 512, in getSqlVectorsOct 7 06:35:13 ferda weewx[27420]: **** aggregate_type, aggregate_interval)Oct 7 06:35:13 ferda weewx[27420]: **** File "/usr/share/weewx/weewx/manager.py", line 758, in _getSqlVectorsOct 7 06:35:13 ferda weewx[27420]: **** for _rec in _cursor.execute(sql_str, (startstamp, stopstamp)):Oct 7 06:35:13 ferda weewx[27420]: **** File "/usr/share/weewx/weedb/mysql.py", line 47, in guarded_fnOct 7 06:35:13 ferda weewx[27420]: **** raise klass(e)Oct 7 06:35:13 ferda weewx[27420]: **** DatabaseError: (2006, 'MySQL server has gone away')Oct 7 06:35:13 ferda weewx[27420]: **** Generator terminatedOct 7 06:35:13 ferda weewx[27420]: copygenerator: copied 0 files to /home/www/weewxOct 7 06:36:12 ferda weewx[27420]: engine: Shutting down StdReport threadOct 7 06:36:12 ferda weewx[27420]: engine: Caught unrecoverable exception in engine:Oct 7 06:36:12 ferda weewx[27420]: **** (2006, 'MySQL server has gone away')Oct 7 06:36:12 ferda weewx[27420]: **** Traceback (most recent call last):Oct 7 06:36:12 ferda weewx[27420]: **** File "/usr/share/weewx/weewx/engine.py", line 871, in mainOct 7 06:36:12 ferda weewx[27420]: **** engine.run()Oct 7 06:36:12 ferda weewx[27420]: **** File "/usr/share/weewx/weewx/engine.py", line 190, in runOct 7 06:36:12 ferda weewx[27420]: **** self.dispatchEvent(weewx.Event(weewx.NEW_LOOP_PACKET, packet=packet))Oct 7 06:36:12 ferda weewx[27420]: **** File "/usr/share/weewx/weewx/engine.py", line 223, in dispatchEventOct 7 06:36:12 ferda weewx[27420]: **** callback(event)Oct 7 06:36:12 ferda weewx[27420]: **** File "/usr/share/weewx/weewx/wxservices.py", line 45, in new_loop_packetOct 7 06:36:12 ferda weewx[27420]: **** self.calc.do_calculations(event.packet, 'loop')Oct 7 06:36:12 ferda weewx[27420]: **** File "/usr/share/weewx/weewx/wxservices.py", line 189, in do_calculationsOct 7 06:36:12 ferda weewx[27420]: **** getattr(self, 'calc_' + obs)(data_us, data_type)Oct 7 06:36:12 ferda weewx[27420]: **** File "/usr/share/weewx/weewx/wxservices.py", line 232, in calc_pressureOct 7 06:36:12 ferda weewx[27420]: **** data['dateTime'], interval)Oct 7 06:36:12 ferda weewx[27420]: **** File "/usr/share/weewx/weewx/wxservices.py", line 434, in _get_temperature_12hOct 7 06:36:12 ferda weewx[27420]: **** record = dbmanager.getRecord(ts12, max_delta=self.max_delta_12h)Oct 7 06:36:12 ferda weewx[27420]: **** File "/usr/share/weewx/weewx/manager.py", line 369, in getRecordOct 7 06:36:12 ferda weewx[27420]: **** (time_start_ts, time_stop_ts, timestamp))Oct 7 06:36:12 ferda weewx[27420]: **** File "/usr/share/weewx/weedb/mysql.py", line 47, in guarded_fnOct 7 06:36:12 ferda weewx[27420]: **** raise klass(e)Oct 7 06:36:12 ferda weewx[27420]: **** DatabaseError: (2006, 'MySQL server has gone away')Oct 7 06:36:12 ferda weewx[27420]: **** Exiting.* Report engine was affected by this restart by log records. As I see, there is same silence in log file after engine MySQL error, which is come even MySQL was back UP. When I run restart, there I see, than WeeWx process 23732 was running: *root@ferda:/etc/rc2.d# systemctl status weewx● weewx.service - LSB: weewx weather system Loaded: loaded (/etc/init.d/weewx) Active: active (running) since Sat 2017-10-07 07:01:28 CEST; 33s ago Process: 23732 ExecStop=/etc/init.d/weewx stop (code=exited, status=0/SUCCESS) Process: 23742 ExecStart=/etc/init.d/weewx start (code=exited, status=0/SUCCESS) CGroup: /system.slice/weewx.service └─23755 python /usr/bin/weewxd --daemon --pidfile=/var/run/weewx.pid /etc/weewx/weewx.conf* Do you think, that it is my fault somewhere in settings or driver code? Where should I start to check what is not correct? Thank you for your help? Frantisek Dne pátek 6. října 2017 1:15:10 UTC+2 Tom Keffer napsal(a): > > Any number of errors will cause WeeWX to "start from the top," that is, > essentially, reboot the internal engine. One such error is a database > error. Part of the rebooting process is to kill the reporting thread. > > So, my guess is that your MySQL database went offline for some reason. > This caused the internal engine to kill the reporting thread, then do a > restart. > > I don't know why your MySQL database went offline --- you'll have to > search for clues elsewhere. Try its error log (found in /var/log/mysql > for Debian systems). For systemd versions you can try "journalctl -u mysql" > for clues. > > -tk > > On Thu, Oct 5, 2017 at 11:13 AM, fpb <[email protected] <javascript:>> > wrote: > >> Hi All. >> >> I am sorry for this question, but I am stuck in solving this problem. >> >> I had been written driver for my IP and XML based PWS. Due to some >> troubles in my network I am loosing connection to PWS. >> >> I tried to manage this situation with goal "When PWS is back online all >> data will be stored to WEEWX without user action." >> Driver using GenArchiveRecords since from first version witht correct >> management of missing data (TS_SINCE), but there was not done any better >> exception handling. I have been tried add exception handling described in >> docs and examples, but I am stuck in this situation, that "engine: Shutting >> down StdReport thread" comming and I do not have any real idea why. My >> guess is because of MySQL DB has closed session and can not be reconnected. >> >> Can give me somebody some hints, how to handle exceptions for lost >> connections to PWS, that it will continue grab data from PWS when it come >> back online (grabing from first missing data works correctly)? >> >> Thank you anybody who can help me to understand and tips how to handle it. >> >> >> >> Below is last part of log. First part is my driver handling of lost >> connection, then after more then 2 days engine come with StdReport + MySQL >> exception. Behavior is that some times fails in 1 day somtimes in 2 or 3 >> days. >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> *LOG:Oct 3 7:06:15 ferda weewx[7865]: wanet3: from >> http://me13.lbn.cz/xml.xml <http://me13.lbn.cz/xml.xml> download failed >> attempt 1: 5 <urlopen error timed out> . . . Oct 5 18:38:59 ferda >> weewx[7865]: wanet3: from http://me13.lbn.cz/xml.xml >> <http://me13.lbn.cz/xml.xml> download failed attempt 1: 5 <urlopen error >> timed out>Oct 5 18:39:39 ferda weewx[7865]: wanet3: from >> http://me13.lbn.cz/xml.xml <http://me13.lbn.cz/xml.xml> download failed >> attempt 2: 5 <urlopen error timed out>Oct 5 18:40:19 ferda weewx[7865]: >> wanet3: from http://me13.lbn.cz/xml.xml <http://me13.lbn.cz/xml.xml> >> download failed attempt 3: 5 <urlopen error timed out>Oct 5 18:40:59 ferda >> weewx[7865]: wanet3: from http://me13.lbn.cz/xml.xml >> <http://me13.lbn.cz/xml.xml> download failed attempt 4: 5 <urlopen error >> timed out>Oct 5 18:41:40 ferda weewx[7865]: wanet3: from >> http://me13.lbn.cz/xml.xml <http://me13.lbn.cz/xml.xml> download failed >> attempt 5: 5 <urlopen error timed out>Oct 5 18:42:10 ferda weewx[7865]: >> wanet3: download failed after 5 triesOct 5 18:43:20 ferda weewx[7865]: >> wanet3: from http://me13.lbn.cz/xml.xml <http://me13.lbn.cz/xml.xml> >> download failed attempt 1: 5 <urlopen error timed out>Oct 5 18:44:00 ferda >> weewx[7865]: wanet3: from http://me13.lbn.cz/xml.xml >> <http://me13.lbn.cz/xml.xml> download failed attempt 2: 5 <urlopen error >> timed out>Oct 5 18:44:40 ferda weewx[7865]: wanet3: from >> http://me13.lbn.cz/xml.xml <http://me13.lbn.cz/xml.xml> download failed >> attempt 3: 5 <urlopen error timed out>Oct 5 18:45:20 ferda weewx[7865]: >> wanet3: from http://me13.lbn.cz/xml.xml <http://me13.lbn.cz/xml.xml> >> download failed attempt 4: 5 <urlopen error timed out>Oct 5 18:46:00 ferda >> weewx[7865]: wanet3: from http://me13.lbn.cz/xml.xml >> <http://me13.lbn.cz/xml.xml> download failed attempt 5: 5 <urlopen error >> timed out>Oct 5 18:46:30 ferda weewx[7865]: wanet3: download failed after >> 5 triesOct 5 18:47:30 ferda weewx[7865]: engine: Shutting down StdReport >> threadOct 5 18:47:30 ferda weewx[7865]: engine: Caught unrecoverable >> exception in engine:Oct 5 18:47:30 ferda weewx[7865]: **** (2006, >> 'MySQL server has gone away')Oct 5 18:47:30 ferda weewx[7865]: **** >> Traceback (most recent call last):Oct 5 18:47:30 ferda weewx[7865]: >> **** File "/usr/share/weewx/weewx/engine.py", line 871, in mainOct 5 >> 18:47:30 ferda weewx[7865]: **** engine.run()Oct 5 18:47:30 ferda >> weewx[7865]: **** File "/usr/share/weewx/weewx/engine.py", line 190, >> in runOct 5 18:47:30 ferda weewx[7865]: **** >> self.dispatchEvent(weewx.Event(weewx.NEW_LOOP_PACKET, packet=packet))Oct 5 >> 18:47:30 ferda weewx[7865]: **** File >> "/usr/share/weewx/weewx/engine.py", line 223, in dispatchEventOct 5 >> 18:47:30 ferda weewx[7865]: **** callback(event)Oct 5 18:47:30 >> ferda weewx[7865]: **** File "/usr/share/weewx/weewx/wxservices.py", >> line 45, in new_loop_packetOct 5 18:47:30 ferda weewx[7865]: **** >> self.calc.do_calculations(event.packet, 'loop')Oct 5 18:47:30 ferda >> weewx[7865]: **** File "/usr/share/weewx/weewx/wxservices.py", line >> 189, in do_calculationsOct 5 18:47:30 ferda weewx[7865]: **** >> getattr(self, 'calc_' + obs)(data_us, data_type)Oct 5 18:47:30 ferda >> weewx[7865]: **** File "/usr/share/weewx/weewx/wxservices.py", line >> 232, in calc_pressureOct 5 18:47:30 ferda weewx[7865]: **** >> data['dateTime'], interval)Oct 5 18:47:30 ferda weewx[7865]: **** >> File "/usr/share/weewx/weewx/wxservices.py", line 434, in >> _get_temperature_12hOct 5 18:47:30 ferda weewx[7865]: **** record >> = dbmanager.getRecord(ts12, max_delta=self.max_delta_12h)Oct 5 18:47:30 >> ferda weewx[7865]: **** File "/usr/share/weewx/weewx/manager.py", >> line 369, in getRecordOct 5 18:47:30 ferda weewx[7865]: **** >> (time_start_ts, time_stop_ts, timestamp))Oct 5 18:47:30 ferda >> weewx[7865]: **** File "/usr/share/weewx/weedb/mysql.py", line 47, >> in guarded_fnOct 5 18:47:30 ferda weewx[7865]: **** raise >> klass(e)Oct 5 18:47:30 ferda weewx[7865]: **** DatabaseError: (2006, >> 'MySQL server has gone away')Oct 5 18:47:30 ferda weewx[7865]: **** >> Exiting.* >> >> >
