After moving the ignore lines, I am happy to report that the time problem
is solved; that is, it is no longer rejecting the topic. However, it is
still not making it into the database. I think this is because it is not
converting from snow/snowDepth to snowDepth. See attached syslog extract
and weewx.conf for section where topic is configured. Is there a mistake in
this topic configuration?
On Sunday, October 11, 2020 at 11:26:27 AM UTC-6 [email protected] wrote:
> I've run both extensions off and on, so I don't think there is any
> conflict. The root-root cause is time skew between your pi and weather
> station and MQTTSubscribe being a bit overly aggressive on its quality
> control. Setting ignore_start_time and ignore_end_time should turn this off
> completely. Since the incoming data has no dateTime, this seems the most
> logical approach.
>
> I noticed in the latest attached config, those options are in the wrong
> place. Move them under [[topics]] after
> unit_system = US
> ignore_start_time = true
> ignore_end_time = true
> Also, Ignore_end_time has a capital "I".
> If it is still not working, please attach the log from startup. That way
> we can see how the options are being processed.
> I think we are real close. -rich
>
> On Sunday, 11 October 2020 at 12:20:32 UTC-4 [email protected] wrote:
>
>> I used wee_extension to uninstall the old version then install the new. I
>> used the configuration as shown in the attached file. When I publish to a
>> topic, Topic Manager still ignores it as outside of interval (syslog
>> extract in attached file).
>>
>> I'm thinking that MQTTSubscribe is not compatible with the mqtt
>> extension. I note that MQTTSubscribe is publishing records of data that it
>> is not subscribed to, but which are published by mqtt. Have you tried
>> running MQTTSubscribe and mqtt together?
>>
>> On Saturday, October 10, 2020 at 5:10:34 PM UTC-6 [email protected]
>> wrote:
>>
>>> I created a pre-release with the fix. It is here,
>>> https://github.com/bellrichm/WeeWX-MQTTSubscribe/releases/tag/v1.6.2-rc03
>>> Hopefully this is it.
>>> rich
>>> On Saturday, 10 October 2020 at 10:15:29 UTC-4 [email protected]
>>> wrote:
>>>
>>>> OK, thanks for your help.
>>>>
>>>> On Saturday, October 10, 2020 at 7:22:18 AM UTC-6 [email protected]
>>>> wrote:
>>>>
>>>>> Thanks for the log. Looks like there is a bug with the
>>>>> ignore_start_time option. Fix looks easy, I just need to make sure there
>>>>> are no side effects.
>>>>> I’ll let you know when the fix is available.
>>>>> - rich
>>>>>
>>>>> On Friday, 9 October 2020 at 22:55:11 UTC-4 [email protected]
>>>>> wrote:
>>>>>
>>>>>> I added those this way:
>>>>>>
>>>>>> # The topics to subscribe to.
>>>>>> [[topics]]
>>>>>> # Units for MQTT payloads without unit value.
>>>>>> # Valid values: US, METRIC, METRICWX
>>>>>> # Default is: US
>>>>>> unit_system = US
>>>>>> ignore_start_time = true
>>>>>> Ignore_end_time = true
>>>>>>
>>>>>> but I still get this:
>>>>>>
>>>>>> Oct 9 20:38:29 raspberrypi weewx[30461] INFO weewx.restx: MQTT:
>>>>>> Published record 2020-10-09 20:38:47 MDT (1602297527)
>>>>>> Oct 9 20:38:30 raspberrypi weewx[30461] DEBUG user.MQTTSubscribe:
>>>>>> (Service) MessageCallbackProvider data-> incoming topic: snow/snowDepth,
>>>>>> QOS: 0, retain: 0, payload: b'7'
>>>>>> Oct 9 20:38:30 raspberrypi weewx[30461] DEBUG user.MQTTSubscribe:
>>>>>> (Service) TopicManager data-> incoming snow/snowDepth: snow/snowDepth:
>>>>>> 7.0
>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] DEBUG user.MQTTSubscribe:
>>>>>> (Service) TopicManager data-> outgoing snow/snowDepth: dateTime:
>>>>>> 1602297510.2596123, snow/snowDepth: 7.0, usUnits: 1
>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] INFO user.MQTTSubscribe:
>>>>>> (Service) TopicManager ignoring record outside of interval
>>>>>> 1602297510.259612 1602297529.000000 1602297510.259612 dateTime:
>>>>>> 1602297510.2596123, snow/snowDepth: 7.0, usUnits: 1
>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] DEBUG user.MQTTSubscribe:
>>>>>> (Service) TopicManager data-> outgoing accumulated snow/snowDepth:
>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] DEBUG user.MQTTSubscribe:
>>>>>> (Service) data-> final packet is 2020-10-09 20:38:49 MDT (1602297529):
>>>>>> avg_distance: 0, dateTime: 1602297529, lightning_strikes: 0,
>>>>>> outHumidity:
>>>>>> 23, outTemp: 13.34, outTempBatteryStatus: 2.919, pressure: 744.7,
>>>>>> usUnits:
>>>>>> 17
>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] INFO weewx.restx: MQTT:
>>>>>> Published record 2020-10-09 20:38:49 MDT (1602297529)
>>>>>> Oct 9 20:38:32 raspberrypi weewx[30461] DEBUG user.MQTTSubscribe:
>>>>>> (Service) data-> final packet is 2020-10-09 20:38:50 MDT (1602297530):
>>>>>> dateTime: 1602297530, usUnits: 17, windDir: 336, windSpeed: 0.72
>>>>>> Oct 9 20:38:32 raspberrypi weewx[30461] INFO weewx.restx: MQTT:
>>>>>> Published record 2020-10-09 20:38:50 MDT (1602297530)
>>>>>>
>>>>>> On Friday, October 9, 2020 at 5:13:41 PM UTC-6 [email protected]
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>> Ah, crap. This has to do with an attempt to quality control data by
>>>>>>> time. Next major release, I really need to rework it...
>>>>>>> Since you have individual payloads (no timestamp from the origin)
>>>>>>> add the following under [[topics]]
>>>>>>> ignore_start_time = true
>>>>>>> Ignore_end_time = true
>>>>>>>
>>>>>>> We are getting close. Sorry for the pain.
>>>>>>> rich
>>>>>>>
>>>>>>>
>>>>>>> On Friday, 9 October 2020 at 18:36:27 UTC-4 [email protected]
>>>>>>> wrote:
>>>>>>>
>>>>>>>> syslog extract posted. weewx is loading the service, though it also
>>>>>>>> seems to subscribe to the data which MQTT is publishing from weewx.
>>>>>>>> the
>>>>>>>> second section shows that subscribe receives the data I published from
>>>>>>>> the
>>>>>>>> terminal (6 inches of snow), but rejects it as outside of interval. Is
>>>>>>>> there an entry to be made somewhere to sync incoming data? I note that
>>>>>>>> another extension I use (GW1000) needed dateTime = datetime in a field
>>>>>>>> map
>>>>>>>> to work properly.
>>>>>>>>
>>>>>>>> On Friday, October 9, 2020 at 2:54:56 PM UTC-6 [email protected]
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> That could be simplified, but looks like it should work. The
>>>>>>>>> quickest and easiest way to proceed is to set debug to 1, restart
>>>>>>>>> WeeWX,
>>>>>>>>> let it run through an archive cycle during which you published to
>>>>>>>>> that
>>>>>>>>> topic and then post the log.
>>>>>>>>>
>>>>>>>>> Re: snowBatteryStatus - The units config option is only needed if
>>>>>>>>> the field units do not match the units expected by the unit_system.
>>>>>>>>> So
>>>>>>>>> eliminating will at least get the data in the DB.
>>>>>>>>> Note, units is not needed for snowDepth because inches is the
>>>>>>>>> Units for that field in the US unit_system.
>>>>>>>>> rich
>>>>>>>>> On Friday, 9 October 2020 at 14:23:01 UTC-4 [email protected]
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I've attached a syslog extract showing where weewx crashed. it
>>>>>>>>>> seems that "volt" is an invalid unit for my topic. i don't know why
>>>>>>>>>> but for
>>>>>>>>>> now I commented out that topic and its parameters. Now, weewx will
>>>>>>>>>> continue
>>>>>>>>>> running when subscribe is enabled, but subscribed topics are not
>>>>>>>>>> being
>>>>>>>>>> posted to the database. Here is the first topic in weewx.conf:
>>>>>>>>>>
>>>>>>>>>> # The first topic
>>>>>>>>>> # MQTT Topic
>>>>>>>>>> [[[snow/snowDepth]]]
>>>>>>>>>> # MQTT name
>>>>>>>>>> [[[[snowDepth]]]]
>>>>>>>>>> # weewx name
>>>>>>>>>> name = snowDepth
>>>>>>>>>> ignore = false
>>>>>>>>>> contains_total = false
>>>>>>>>>> conversion_type = float
>>>>>>>>>> units = inch
>>>>>>>>>>
>>>>>>>>>> I used a terminal to publish "6" to snow/snowDepth on Mosquitto.
>>>>>>>>>> Another terminal window command to subscribe to snow/snowDepth
>>>>>>>>>> received the
>>>>>>>>>> "6" but the database entries for snowDepth are null. Is this
>>>>>>>>>> configuration
>>>>>>>>>> of topics still not correct. Thanks.
>>>>>>>>>> On Tuesday, October 6, 2020 at 1:43:31 PM UTC-6 Timothy Buchanan
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks, Rich. I'll try it when I get back. I'll be in our second
>>>>>>>>>>> home for a few days: going to the gun range and the
>>>>>>>>>>> clothing-optional
>>>>>>>>>>> resort (not at the same time!).
>>>>>>>>>>>
>>>>>>>>>>> On Tuesday, October 6, 2020 at 1:19:26 PM UTC-6
>>>>>>>>>>> [email protected] wrote:
>>>>>>>>>>>
>>>>>>>>>>>> So, the [[[first/topic]]] are meant to be replaced with the
>>>>>>>>>>>> actual topic. So it would be something like this
>>>>>>>>>>>> ```
>>>>>>>>>>>> [[Topics]]
>>>>>>>>>>>> [[[topic name that snowDepth is published on]]]
>>>>>>>>>>>> [[[[topic name that snowDepth is published on]]]]
>>>>>>>>>>>> name = snowDepth
>>>>>>>>>>>> [[[topic name that snowRate is published on]]]
>>>>>>>>>>>> [[[[topic name that snowRate is published on]]]]
>>>>>>>>>>>> name = snowRate
>>>>>>>>>>>> ```
>>>>>>>>>>>> The duplication is an artifact of dealing with json and keyword
>>>>>>>>>>>> payloads. The ```use_topic_as_fieldname``` option can be used
>>>>>>>>>>>> to make the config a bit prettier.
>>>>>>>>>>>> ```
>>>>>>>>>>>> [[Topics]]
>>>>>>>>>>>> use_topic_as_fieldname = true
>>>>>>>>>>>> [[[topic name that snowDepth is published on]]]
>>>>>>>>>>>> name = snowDepth
>>>>>>>>>>>> [[[topic name that snowRate is published on]]]
>>>>>>>>>>>> name = snowRate
>>>>>>>>>>>> ```
>>>>>>>>>>>> Note, if snowDepth is actually published on the topic
>>>>>>>>>>>> snowDepth, the ```name``` option can be left off.
>>>>>>>>>>>> I don’t think that you want to set contains_total=true for
>>>>>>>>>>>> snowDepth. This is used when the field contains a total and it
>>>>>>>>>>>> needs to be
>>>>>>>>>>>> converted into an increment for WeeWX.
>>>>>>>>>>>>
>>>>>>>>>>>> I’ll work on clarifying the wiki.
>>>>>>>>>>>>
>>>>>>>>>>>> With that said, it shouldn’t have broken WeeWX. If you are up
>>>>>>>>>>>> for it, before changing the config, setting debug=1, restarting
>>>>>>>>>>>> WeeWX for a
>>>>>>>>>>>> couple of archive intervals and attaching the log would be
>>>>>>>>>>>> appreciated .
>>>>>>>>>>>>
>>>>>>>>>>>> rich
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tuesday, 6 October 2020 at 13:28:34 UTC-4
>>>>>>>>>>>> [email protected] wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Attached is the extension material that I put into weewx.conf.
>>>>>>>>>>>>> But when I set enable = true, weewx stops archiving data. Is
>>>>>>>>>>>>> there an error
>>>>>>>>>>>>> in this configuration, or could subscribe be incompatible with
>>>>>>>>>>>>> another
>>>>>>>>>>>>> service? I'm using the Weatherflowudp driver with mqtt and GW1000
>>>>>>>>>>>>> extensions.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tuesday, October 6, 2020 at 9:17:07 AM UTC-6 Timothy
>>>>>>>>>>>>> Buchanan wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks, Rich, I should be able to edit weewx.conf based on
>>>>>>>>>>>>>> the example at the bottom of that page.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I am using an ESP8266 board with an ultrasonic sensor and a
>>>>>>>>>>>>>> temperature sensor (to calibrate the speed of sound), and
>>>>>>>>>>>>>> programming in
>>>>>>>>>>>>>> the Arduino IDE. I'll 3D print a case and mount it above my
>>>>>>>>>>>>>> deck. The
>>>>>>>>>>>>>> materials cost about $15.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'd be happy to post the code here, under a new topic, once I
>>>>>>>>>>>>>> get it working.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Timothy
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tuesday, October 6, 2020 at 8:53:45 AM UTC-6
>>>>>>>>>>>>>> [email protected] wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What type of sensor are you using for measuring snow depth?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/58f40395-8c5c-4193-aee4-130790a66512n%40googlegroups.com.
Oct 11 14:33:02 raspberrypi weewx[10390] INFO weewx.restx: MQTT: Published
record 2020-10-11 14:33:07 MDT (1602448387)
Oct 11 14:33:04 raspberrypi weewx[10390] DEBUG gw1000: user.gw1000: Next update
in 60 seconds
Oct 11 14:33:04 raspberrypi weewx[10390] DEBUG user.MQTTSubscribe: (Service)
data-> final packet is 2020-10-11 14:33:10 MDT (1602448390): dateTime:
1602448390, extraHumid1: 20, extraHumid2: 24, extraTemp1: 23.8, extraTemp2:
18.5, inHumidity: 20, inTemp: 24.0, soilMoist1: 28, soilMoist2: 42, usUnits:
17, windDir: 263, windSpeed: 1.21
Oct 11 14:33:04 raspberrypi weewx[10390] INFO weewx.restx: MQTT: Published
record 2020-10-11 14:33:10 MDT (1602448390)
Oct 11 14:33:06 raspberrypi weewx[10390] DEBUG user.MQTTSubscribe: (Service)
data-> final packet is 2020-10-11 14:33:12 MDT (1602448392): dateTime:
1602448392, lux: 74292, radiation: 619, rain: 0.0, usUnits: 17, UV: 6.29,
windBatteryStatus: 3.491, windGust: 5.81
Oct 11 14:33:06 raspberrypi weewx[10390] INFO weewx.restx: MQTT: Published
record 2020-10-11 14:33:12 MDT (1602448392)
Oct 11 14:33:09 raspberrypi weewx[10390] DEBUG user.MQTTSubscribe: (Service)
data-> final packet is 2020-10-11 14:33:13 MDT (1602448393): dateTime:
1602448393, usUnits: 17, windDir: 249, windSpeed: 1.52
Oct 11 14:33:09 raspberrypi weewx[10390] INFO weewx.restx: MQTT: Published
record 2020-10-11 14:33:13 MDT (1602448393)
Oct 11 14:33:10 raspberrypi weewx[10390] DEBUG user.MQTTSubscribe: (Service)
MessageCallbackProvider data-> incoming topic: snow/snowDepth, QOS: 0, retain:
0, payload: b'3.5'
Oct 11 14:33:10 raspberrypi weewx[10390] DEBUG user.MQTTSubscribe: (Service)
TopicManager data-> incoming snow/snowDepth: snow/snowDepth: 3.5
Oct 11 14:33:11 raspberrypi weewx[10390] DEBUG user.MQTTSubscribe: (Service)
TopicManager data-> outgoing snow/snowDepth: dateTime: 1602448390.4433427,
snow/snowDepth: 3.5, usUnits: 1
Oct 11 14:33:11 raspberrypi weewx[10390] DEBUG user.MQTTSubscribe: (Service)
TopicManager data-> outgoing accumulated snow/snowDepth: dateTime: 1602448396,
snow/snowDepth: 3.5, usUnits: 17
Oct 11 14:33:11 raspberrypi weewx[10390] DEBUG user.MQTTSubscribe: (Service)
data-> final packet is 2020-10-11 14:33:16 MDT (1602448396): dateTime:
1602448396, snow/snowDepth: 3.5, usUnits: 17, windDir: 239, windSpeed: 3.17
Oct 11 14:33:11 raspberrypi weewx[10390] INFO weewx.restx: MQTT: Published
record 2020-10-11 14:33:16 MDT (1602448396)
Oct 11 14:33:14 raspberrypi weewx[10390] DEBUG user.MQTTSubscribe: (Service)
data-> final packet is 2020-10-11 14:33:19 MDT (1602448399): dateTime:
1602448399, usUnits: 17, windDir: 255, windSpeed: 2.28
Oct 11 14:33:14 raspberrypi weewx[10390] INFO weewx.restx: MQTT: Published
record 2020-10-11 14:33:19 MDT (1602448399)
Oct 11 14:33:17 raspberrypi weewx[10390] DEBUG user.MQTTSubscribe: (Service)
data-> final packet is 2020-10-11 14:33:22 MDT (1602448402): dateTime:
1602448402, usUnits: 17, windDir: 235, windSpeed: 2.77
Oct 11 14:33:17 raspberrypi weewx[10390] INFO weewx.restx: MQTT: Published
record 2020-10-11 14:33:22 MDT (1602448402)
# The first topic
# MQTT Topic
[[[snow/snowDepth]]]
# MQTT name
[[[[snowDepth]]]]
# weewx name
name = snowDepth
ignore = false
contains_total = false
conversion_type = float
units = inch
