Currently, MQTTSubscribe does require that the MQTT data arrives in chronological order. There are two things I think I could do to alleviate this a bit. 1) Currently it requires the data to be in chronological order across all topics. I think I could fix it so that it only needs to be in order within a topic. 2) “Throw away” the data when it is not in order. This would keep WeeWX running.
Are you subscribing to multiple topics? How have you configured MQTTSubscribe? What version did you install? I’ll look into both options. - Rich On Thu, Jun 20, 2019 at 2:32 PM Patrick Mendiuk <[email protected]> wrote: > I am trying to setup Weewx 3.9.1 with WeatherflowUDP, MQTTSubscribe, MQTT > and the Belchertown 1.0 skin. The platform is Ubuntu 18.04 LTS running on > an Atom D2700 with 4G RAM and 256G OCZ SSD. The MQTT Mosquitto broker is > running on the same machine. > > Previously, I was running Weewx, WeatherflowUDP and MQTT for 2 or 3 > months. I occasionally would have small gaps in data, but weewx would > continue to run. A couple of days ago I installed MQTTSubscribe to add > some indoor data from a Sonoff TH10 flashed with Tasmota. I have the > Tasmota publishing temp and humidity once a minute. > > Weewx has been stopping due to time stamp error with MQTTSubscribe. I am > wondering if the small gaps in the database I was observing previously is > due to this out of order data just being ignored prior to installing > MQTTSubscribe? > > Jun 20 07:48:02 MITX-6930 weewx[8675]: engine: Main loop exiting. Shutting >> engine down. >> Jun 20 07:48:02 MITX-6930 weewx[8675]: engine: Shutting down StdReport >> thread >> Jun 20 07:48:02 MITX-6930 weewx[8675]: engine: Caught unrecoverable >> exception in engine: >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** start time (1561042080) >> is greater than stop time (1561042079) >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** Traceback (most recent >> call last): >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** File >> "/usr/share/weewx/weewx/engine.py", line 890, in main >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** engine.run() >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** File >> "/usr/share/weewx/weewx/engine.py", line 191, in run >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** >> self.dispatchEvent(weewx.Event(weewx.NEW_LOOP_PACKET, packet=packet)) >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** File >> "/usr/share/weewx/weewx/engine.py", line 224, in dispatchEvent >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** callback(event) >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** File >> "/usr/share/weewx/user/MQTTSubscribe.py", line 609, in new_loop_packet >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** target_data = >> self.subscriber.get_accumulated_data(topic, start_ts, self.end_ts, >> event.packet['usUnits']) >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** File >> "/usr/share/weewx/user/MQTTSubscribe.py", line 529, in get_accumulated_data >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** return >> self.manager.get_accumulated_data(topic, start_ts, end_ts, units) >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** File >> "/usr/share/weewx/user/MQTTSubscribe.py", line 277, in get_accumulated_data >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** accumulator = >> weewx.accum.Accum(weeutil.weeutil.TimeSpan(start_ts, end_ts)) >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** File >> "/usr/share/weewx/weeutil/weeutil.py", line 256, in __new__ >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** raise >> ValueError("start time (%d) is greater than stop time (%d)" % (args[0], >> args[1])) >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** ValueError: start time >> (1561042080) is greater than stop time (1561042079) >> Jun 20 07:48:02 MITX-6930 weewx[8675]: **** Exiting. > > > Jun 20 09:32:42 MITX-6930 weewx[14061]: MQTTSubscribeService: Packet after >> update is: 2019-06-20 09:32:39 PDT (1561048359) dateTime: 1561048359, >> usUnits: 17, windDir: 0, windSpeed: 0.0 >> Jun 20 09:32:42 MITX-6930 weewx[14061]: restx: MQTT: Published record >> 2019-06-20 09:32:39 PDT (1561048359) >> Jun 20 09:32:45 MITX-6930 weewx[14061]: MQTTSubscribeService: Packet >> prior to update is: 2019-06-20 09:32:42 PDT (1561048362) dateTime: >> 1561048362, usUnits: 17, windDir: 0, windSpeed: 0.0 >> Jun 20 09:32:45 MITX-6930 weewx[14061]: MQTTSubscribeService: Processing >> interval: 1561048359.000000 1561048362.000000 >> Jun 20 09:32:45 MITX-6930 weewx[14061]: MQTTSubscribe: TopicManager queue >> size is: 0 >> Jun 20 09:32:45 MITX-6930 weewx[14061]: MQTTSubscribeService: Queue was >> empty >> Jun 20 09:32:45 MITX-6930 weewx[14061]: MQTTSubscribeService: Packet >> after update is: 2019-06-20 09:32:42 PDT (1561048362) dateTime: 1561048362, >> usUnits: 17, windDir: 0, windSpeed: 0.0 >> Jun 20 09:32:45 MITX-6930 weewx[14061]: restx: MQTT: Published record >> 2019-06-20 09:32:42 PDT (1561048362) >> Jun 20 09:32:45 MITX-6930 weewx[14061]: MQTTSubscribeService: Packet >> prior to update is: 2019-06-20 09:32:17 PDT (1561048337) dateTime: >> 1561048337, illuminance: 40134, precipitationType: 0, radiation: 334, rain: >> 0.0, skyBatteryVoltage: 3.37, usUnits: 17, UV: 3.77, windBatteryStatus: >> 3.37, windLull: 0.0 >> Jun 20 09:32:45 MITX-6930 weewx[14061]: MQTTSubscribeService: Processing >> interval: 1561048362.000000 1561048337.000000 >> Jun 20 09:32:45 MITX-6930 weewx[14061]: engine: Main loop exiting. >> Shutting engine down. >> Jun 20 09:32:45 MITX-6930 weewx[14061]: engine: Shutting down StdReport >> thread >> Jun 20 09:32:45 MITX-6930 weewx[14061]: engine: StdReport thread has been >> terminated >> Jun 20 09:32:45 MITX-6930 weewx[14061]: restx: Shut down MQTT thread. >> Jun 20 09:32:45 MITX-6930 weewx[14061]: restx: Shut down Wunderground-RF >> thread. >> Jun 20 09:32:45 MITX-6930 weewx[14061]: MQTTSubscribe: Disconnected with >> result code 0 >> Jun 20 09:32:45 MITX-6930 weewx[14061]: engine: Caught unrecoverable >> exception in engine: >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** start time (1561048362) >> is greater than stop time (1561048337) >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** Traceback (most recent >> call last): >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** File >> "/usr/share/weewx/weewx/engine.py", line 890, in main >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** engine.run() >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** File >> "/usr/share/weewx/weewx/engine.py", line 191, in run >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** >> self.dispatchEvent(weewx.Event(weewx.NEW_LOOP_PACKET, packet=packet)) >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** File >> "/usr/share/weewx/weewx/engine.py", line 224, in dispatchEvent >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** callback(event) >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** File >> "/usr/share/weewx/user/MQTTSubscribe.py", line 609, in new_loop_packet >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** target_data = >> self.subscriber.get_accumulated_data(topic, start_ts, self.end_ts, >> event.packet['usUnits']) >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** File >> "/usr/share/weewx/user/MQTTSubscribe.py", line 529, in get_accumulated_data >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** return >> self.manager.get_accumulated_data(topic, start_ts, end_ts, units) >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** File >> "/usr/share/weewx/user/MQTTSubscribe.py", line 277, in get_accumulated_data >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** accumulator = >> weewx.accum.Accum(weeutil.weeutil.TimeSpan(start_ts, end_ts)) >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** File >> "/usr/share/weewx/weeutil/weeutil.py", line 256, in __new__ >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** raise >> ValueError("start time (%d) is greater than stop time (%d)" % (args[0], >> args[1])) >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** ValueError: start time >> (1561048362) is greater than stop time (1561048337) >> Jun 20 09:32:45 MITX-6930 weewx[14061]: **** Exiting. > > > Jun 20 11:05:35 MITX-6930 weewx[16315]: weatherflowudp: MainThread: raw >> packet: {'serial_number': 'SK-00015052', 'type': 'rapid_wind', 'ob': >> [1561053931, 0.98, 342], 'hub_sn': 'HB-00011764'} >> Jun 20 11:05:35 MITX-6930 weewx[16315]: weatherflowudp: MainThread: >> parsed packet: {'serial_number.SK_00015052.rapid_wind': 'SK-00015052', >> 'type.SK_00015052.rapid_wind': 'rapid_wind', >> 'hub_sn.SK_00015052.rapid_wind': 'HB-00011764', >> 'time_epoch.SK_00015052.rapid_wind': 1561053931, >> 'ob.SK_00015052.rapid_wind': [1561053931, 0.98, 342], >> 'wind_speed.SK_00015052.rapid_wind': 0.98, >> 'wind_direction.SK_00015052.rapid_wind': 342, 'time_epoch': 1561053931} >> Jun 20 11:05:35 MITX-6930 weewx[16315]: weatherflowudp: MainThread: Loop >> packet: {'windDir': 342, 'windSpeed': 0.98, 'usUnits': 17, 'dateTime': >> 1561053931} >> Jun 20 11:05:35 MITX-6930 weewx[16315]: MQTTSubscribeService: Packet >> prior to update is: 2019-06-20 11:05:31 PDT (1561053931) dateTime: >> 1561053931, usUnits: 17, windDir: 342, windSpeed: 0.98 >> Jun 20 11:05:35 MITX-6930 weewx[16315]: MQTTSubscribeService: Processing >> interval: 1561053928.000000 1561053931.000000 >> Jun 20 11:05:35 MITX-6930 weewx[16315]: MQTTSubscribe: TopicManager queue >> size is: 0 >> Jun 20 11:05:35 MITX-6930 weewx[16315]: MQTTSubscribeService: Queue was >> empty >> Jun 20 11:05:35 MITX-6930 weewx[16315]: MQTTSubscribeService: Packet >> after update is: 2019-06-20 11:05:31 PDT (1561053931) dateTime: 1561053931, >> usUnits: 17, windDir: 342, windSpeed: 0.98 >> Jun 20 11:05:35 MITX-6930 weewx[16315]: restx: MQTT: Published record >> 2019-06-20 11:05:31 PDT (1561053931) >> Jun 20 11:05:37 MITX-6930 weewx[16315]: weatherflowudp: MainThread: raw >> packet: {'firmware_revision': 43, 'serial_number': 'SK-00015052', 'type': >> 'obs_sky', 'obs': [[1561053914, 29168, 3.06, 0.0, 0.0, 0.68, 1.52, 7, 3.39, >> 1, 243, None, 0, 3]], 'hub_sn': 'HB-00011764'} >> Jun 20 11:05:37 MITX-6930 weewx[16315]: weatherflowudp: MainThread: >> parsed packet: {'wind_avg.SK_00015052.obs_sky': 0.68, >> 'type.SK_00015052.obs_sky': 'obs_sky', >> 'rain_accumulated.SK_00015052.obs_sky': 0.0, 'hub_sn.SK_00015052.obs_sky': >> 'HB-00011764', 'precipitation_type.SK_00015052.obs_sky': 0, >> 'uv.SK_00015052.obs_sky': 3.06, 'firmware_revision.SK_00015052.obs_sky': >> 43, 'wind_gust.SK_00015052.obs_sky': 1.52, >> 'wind_direction.SK_00015052.obs_sky': 7, >> 'local_day_rain_accumulation.SK_00015052.obs_sky': None, >> 'obs.SK_00015052.obs_sky': [[1561053914, 29168, 3.06, 0.0, 0.0, 0.68, 1.52, >> 7, 3.39, 1, 243, None, 0, 3]], 'wind_sample_interval.SK_00015052.obs_sky': >> 3, 'illuminance.SK_00015052.obs_sky': 29168, >> 'report_interval.SK_00015052.obs_sky': 1, 'time_epoch.SK_00015052.obs_sky': >> 1561053914, 'solar_radiation.SK_00015052.obs_sky': 243, 'time_epoch': >> 1561053914, 'wind_lull.SK_00015052.obs_sky': 0.0, >> 'battery.SK_00015052.obs_sky': 3.39, 'serial_number.SK_00015052.obs_sky': >> 'SK-00015052'} >> Jun 20 11:05:37 MITX-6930 weewx[16315]: weatherflowudp: MainThread: Loop >> packet: {'skyBatteryVoltage': 3.39, 'precipitationType': 0, >> 'windBatteryStatus': 3.39, 'UV': 3.06, 'radiation': 243, 'rain': 0.0, >> 'dateTime': 1561053914, 'illuminance': 29168, 'windLull': 0.0, 'usUnits': >> 17} >> Jun 20 11:05:37 MITX-6930 weewx[16315]: MQTTSubscribeService: Packet >> prior to update is: 2019-06-20 11:05:14 PDT (1561053914) dateTime: >> 1561053914, illuminance: 29168, precipitationType: 0, radiation: 243, rain: >> 0.0, skyBatteryVoltage: 3.39, usUnits: 17, UV: 3.06, windBatteryStatus: >> 3.39, windLull: 0.0 >> Jun 20 11:05:37 MITX-6930 weewx[16315]: MQTTSubscribeService: Processing >> interval: 1561053931.000000 1561053914.000000 >> Jun 20 11:05:37 MITX-6930 weewx[16315]: engine: Main loop exiting. >> Shutting engine down. >> Jun 20 11:05:37 MITX-6930 weewx[16315]: engine: Shutting down StdReport >> thread >> Jun 20 11:05:37 MITX-6930 weewx[16315]: engine: StdReport thread has been >> terminated >> Jun 20 11:05:37 MITX-6930 weewx[16315]: restx: Shut down MQTT thread. >> Jun 20 11:05:37 MITX-6930 weewx[16315]: restx: Shut down Wunderground-RF >> thread. >> Jun 20 11:05:37 MITX-6930 weewx[16315]: MQTTSubscribe: Disconnected with >> result code 0 >> Jun 20 11:05:37 MITX-6930 weewx[16315]: engine: Caught unrecoverable >> exception in engine: >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** start time (1561053931) >> is greater than stop time (1561053914) >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** Traceback (most recent >> call last): >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** File >> "/usr/share/weewx/weewx/engine.py", line 890, in main >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** engine.run() >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** File >> "/usr/share/weewx/weewx/engine.py", line 191, in run >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** >> self.dispatchEvent(weewx.Event(weewx.NEW_LOOP_PACKET, packet=packet)) >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** File >> "/usr/share/weewx/weewx/engine.py", line 224, in dispatchEvent >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** callback(event) >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** File >> "/usr/share/weewx/user/MQTTSubscribe.py", line 609, in new_loop_packet >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** target_data = >> self.subscriber.get_accumulated_data(topic, start_ts, self.end_ts, >> event.packet['usUnits']) >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** File >> "/usr/share/weewx/user/MQTTSubscribe.py", line 529, in get_accumulated_data >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** return >> self.manager.get_accumulated_data(topic, start_ts, end_ts, units) >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** File >> "/usr/share/weewx/user/MQTTSubscribe.py", line 277, in get_accumulated_data >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** accumulator = >> weewx.accum.Accum(weeutil.weeutil.TimeSpan(start_ts, end_ts)) >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** File >> "/usr/share/weewx/weeutil/weeutil.py", line 256, in __new__ >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** raise >> ValueError("start time (%d) is greater than stop time (%d)" % (args[0], >> args[1])) >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** ValueError: start time >> (1561053931) is greater than stop time (1561053914) >> Jun 20 11:05:37 MITX-6930 weewx[16315]: **** Exiting. > > > > I really would like to able to include other sensor data beyond the > WeatherFlow sensors. > > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/5eb4dd21-d83a-4585-9d45-2da47626634f%40googlegroups.com > <https://groups.google.com/d/msgid/weewx-user/5eb4dd21-d83a-4585-9d45-2da47626634f%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAG4CQqz7qV02Lup5F_dGftxjWbfge2_%3DehpAO1e9KBD923CcsA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
