See attached.
On Sunday, October 11, 2020 at 5:45:43 PM UTC-6 bell...@gmail.com wrote:
> Sorry, I didn’t get the config quite right. I don’t subscribe to
> ‘individual’ topics, so this is untested, but should work. If it doesn’t,
> I’ll spin up a debug environment to figure it out.
> [[topics]]
> Keep the current settings, unit_system, ignore_start_time,
> ignore_end_time, etc.
>
> # The next option will take the tail end of the topic and use it as the
> MQTT field name
> # So, if the topic is snow/snowDepth, the MQTT field name is snowDepth
> # Only valid when the payload is type ‘individual’
> use_topic_as_fieldname = true
> [[[snow/snowDepth]]]
> # Because snowDepth is the WeeWX name and the defaults work for you,
> nothing else is needed.
> [[[snow/snowRate]]]
> # Not sure, but the defaults should probably work here...
>
> Again, if this doesn’t work; post the log and I’ll spin up a debug
> environment to figure out what is going on.
> rich
> On Sunday, 11 October 2020 at 16:43:58 UTC-4 timothye...@gmail.com wrote:
>
>> After moving the ignore lines, I am happy to report that the time problem
>> is solved; that is, it is no longer rejecting the topic. However, it is
>> still not making it into the database. I think this is because it is not
>> converting from snow/snowDepth to snowDepth. See attached syslog extract
>> and weewx.conf for section where topic is configured. Is there a mistake in
>> this topic configuration?
>>
>> On Sunday, October 11, 2020 at 11:26:27 AM UTC-6 bell...@gmail.com wrote:
>>
>>> I've run both extensions off and on, so I don't think there is any
>>> conflict. The root-root cause is time skew between your pi and weather
>>> station and MQTTSubscribe being a bit overly aggressive on its quality
>>> control. Setting ignore_start_time and ignore_end_time should turn this off
>>> completely. Since the incoming data has no dateTime, this seems the most
>>> logical approach.
>>>
>>> I noticed in the latest attached config, those options are in the wrong
>>> place. Move them under [[topics]] after
>>> unit_system = US
>>> ignore_start_time = true
>>> ignore_end_time = true
>>> Also, Ignore_end_time has a capital "I".
>>> If it is still not working, please attach the log from startup. That way
>>> we can see how the options are being processed.
>>> I think we are real close. -rich
>>>
>>> On Sunday, 11 October 2020 at 12:20:32 UTC-4 timothye...@gmail.com
>>> wrote:
>>>
>>>> I used wee_extension to uninstall the old version then install the new.
>>>> I used the configuration as shown in the attached file. When I publish to
>>>> a
>>>> topic, Topic Manager still ignores it as outside of interval (syslog
>>>> extract in attached file).
>>>>
>>>> I'm thinking that MQTTSubscribe is not compatible with the mqtt
>>>> extension. I note that MQTTSubscribe is publishing records of data that it
>>>> is not subscribed to, but which are published by mqtt. Have you tried
>>>> running MQTTSubscribe and mqtt together?
>>>>
>>>> On Saturday, October 10, 2020 at 5:10:34 PM UTC-6 bell...@gmail.com
>>>> wrote:
>>>>
>>>>> I created a pre-release with the fix. It is here,
>>>>> https://github.com/bellrichm/WeeWX-MQTTSubscribe/releases/tag/v1.6.2-rc03
>>>>> Hopefully this is it.
>>>>> rich
>>>>> On Saturday, 10 October 2020 at 10:15:29 UTC-4 timothye...@gmail.com
>>>>> wrote:
>>>>>
>>>>>> OK, thanks for your help.
>>>>>>
>>>>>> On Saturday, October 10, 2020 at 7:22:18 AM UTC-6 bell...@gmail.com
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks for the log. Looks like there is a bug with the
>>>>>>> ignore_start_time option. Fix looks easy, I just need to make sure
>>>>>>> there
>>>>>>> are no side effects.
>>>>>>> I’ll let you know when the fix is available.
>>>>>>> - rich
>>>>>>>
>>>>>>> On Friday, 9 October 2020 at 22:55:11 UTC-4 timothye...@gmail.com
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I added those this way:
>>>>>>>>
>>>>>>>> # The topics to subscribe to.
>>>>>>>> [[topics]]
>>>>>>>> # Units for MQTT payloads without unit value.
>>>>>>>> # Valid values: US, METRIC, METRICWX
>>>>>>>> # Default is: US
>>>>>>>> unit_system = US
>>>>>>>> ignore_start_time = true
>>>>>>>> Ignore_end_time = true
>>>>>>>>
>>>>>>>> but I still get this:
>>>>>>>>
>>>>>>>> Oct 9 20:38:29 raspberrypi weewx[30461] INFO weewx.restx: MQTT:
>>>>>>>> Published record 2020-10-09 20:38:47 MDT (1602297527)
>>>>>>>> Oct 9 20:38:30 raspberrypi weewx[30461] DEBUG user.MQTTSubscribe:
>>>>>>>> (Service) MessageCallbackProvider data-> incoming topic:
>>>>>>>> snow/snowDepth,
>>>>>>>> QOS: 0, retain: 0, payload: b'7'
>>>>>>>> Oct 9 20:38:30 raspberrypi weewx[30461] DEBUG user.MQTTSubscribe:
>>>>>>>> (Service) TopicManager data-> incoming snow/snowDepth: snow/snowDepth:
>>>>>>>> 7.0
>>>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] DEBUG user.MQTTSubscribe:
>>>>>>>> (Service) TopicManager data-> outgoing snow/snowDepth: dateTime:
>>>>>>>> 1602297510.2596123, snow/snowDepth: 7.0, usUnits: 1
>>>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] INFO user.MQTTSubscribe:
>>>>>>>> (Service) TopicManager ignoring record outside of interval
>>>>>>>> 1602297510.259612 1602297529.000000 1602297510.259612 dateTime:
>>>>>>>> 1602297510.2596123, snow/snowDepth: 7.0, usUnits: 1
>>>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] DEBUG user.MQTTSubscribe:
>>>>>>>> (Service) TopicManager data-> outgoing accumulated snow/snowDepth:
>>>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] DEBUG user.MQTTSubscribe:
>>>>>>>> (Service) data-> final packet is 2020-10-09 20:38:49 MDT (1602297529):
>>>>>>>> avg_distance: 0, dateTime: 1602297529, lightning_strikes: 0,
>>>>>>>> outHumidity:
>>>>>>>> 23, outTemp: 13.34, outTempBatteryStatus: 2.919, pressure: 744.7,
>>>>>>>> usUnits:
>>>>>>>> 17
>>>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] INFO weewx.restx: MQTT:
>>>>>>>> Published record 2020-10-09 20:38:49 MDT (1602297529)
>>>>>>>> Oct 9 20:38:32 raspberrypi weewx[30461] DEBUG user.MQTTSubscribe:
>>>>>>>> (Service) data-> final packet is 2020-10-09 20:38:50 MDT (1602297530):
>>>>>>>> dateTime: 1602297530, usUnits: 17, windDir: 336, windSpeed: 0.72
>>>>>>>> Oct 9 20:38:32 raspberrypi weewx[30461] INFO weewx.restx: MQTT:
>>>>>>>> Published record 2020-10-09 20:38:50 MDT (1602297530)
>>>>>>>>
>>>>>>>> On Friday, October 9, 2020 at 5:13:41 PM UTC-6 bell...@gmail.com
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ah, crap. This has to do with an attempt to quality control data
>>>>>>>>> by time. Next major release, I really need to rework it...
>>>>>>>>> Since you have individual payloads (no timestamp from the origin)
>>>>>>>>> add the following under [[topics]]
>>>>>>>>> ignore_start_time = true
>>>>>>>>> Ignore_end_time = true
>>>>>>>>>
>>>>>>>>> We are getting close. Sorry for the pain.
>>>>>>>>> rich
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Friday, 9 October 2020 at 18:36:27 UTC-4 timothye...@gmail.com
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> syslog extract posted. weewx is loading the service, though it
>>>>>>>>>> also seems to subscribe to the data which MQTT is publishing from
>>>>>>>>>> weewx.
>>>>>>>>>> the second section shows that subscribe receives the data I
>>>>>>>>>> published from
>>>>>>>>>> the terminal (6 inches of snow), but rejects it as outside of
>>>>>>>>>> interval. Is
>>>>>>>>>> there an entry to be made somewhere to sync incoming data? I note
>>>>>>>>>> that
>>>>>>>>>> another extension I use (GW1000) needed dateTime = datetime in a
>>>>>>>>>> field map
>>>>>>>>>> to work properly.
>>>>>>>>>>
>>>>>>>>>> On Friday, October 9, 2020 at 2:54:56 PM UTC-6 bell...@gmail.com
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> That could be simplified, but looks like it should work. The
>>>>>>>>>>> quickest and easiest way to proceed is to set debug to 1, restart
>>>>>>>>>>> WeeWX,
>>>>>>>>>>> let it run through an archive cycle during which you published to
>>>>>>>>>>> that
>>>>>>>>>>> topic and then post the log.
>>>>>>>>>>>
>>>>>>>>>>> Re: snowBatteryStatus - The units config option is only needed
>>>>>>>>>>> if the field units do not match the units expected by the
>>>>>>>>>>> unit_system. So
>>>>>>>>>>> eliminating will at least get the data in the DB.
>>>>>>>>>>> Note, units is not needed for snowDepth because inches is the
>>>>>>>>>>> Units for that field in the US unit_system.
>>>>>>>>>>> rich
>>>>>>>>>>> On Friday, 9 October 2020 at 14:23:01 UTC-4
>>>>>>>>>>> timothye...@gmail.com wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I've attached a syslog extract showing where weewx crashed. it
>>>>>>>>>>>> seems that "volt" is an invalid unit for my topic. i don't know
>>>>>>>>>>>> why but for
>>>>>>>>>>>> now I commented out that topic and its parameters. Now, weewx will
>>>>>>>>>>>> continue
>>>>>>>>>>>> running when subscribe is enabled, but subscribed topics are not
>>>>>>>>>>>> being
>>>>>>>>>>>> posted to the database. Here is the first topic in weewx.conf:
>>>>>>>>>>>>
>>>>>>>>>>>> # The first topic
>>>>>>>>>>>> # MQTT Topic
>>>>>>>>>>>> [[[snow/snowDepth]]]
>>>>>>>>>>>> # MQTT name
>>>>>>>>>>>> [[[[snowDepth]]]]
>>>>>>>>>>>> # weewx name
>>>>>>>>>>>> name = snowDepth
>>>>>>>>>>>> ignore = false
>>>>>>>>>>>> contains_total = false
>>>>>>>>>>>> conversion_type = float
>>>>>>>>>>>> units = inch
>>>>>>>>>>>>
>>>>>>>>>>>> I used a terminal to publish "6" to snow/snowDepth on
>>>>>>>>>>>> Mosquitto. Another terminal window command to subscribe to
>>>>>>>>>>>> snow/snowDepth
>>>>>>>>>>>> received the "6" but the database entries for snowDepth are null.
>>>>>>>>>>>> Is this
>>>>>>>>>>>> configuration of topics still not correct. Thanks.
>>>>>>>>>>>> On Tuesday, October 6, 2020 at 1:43:31 PM UTC-6 Timothy
>>>>>>>>>>>> Buchanan wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks, Rich. I'll try it when I get back. I'll be in our
>>>>>>>>>>>>> second home for a few days: going to the gun range and the
>>>>>>>>>>>>> clothing-optional resort (not at the same time!).
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tuesday, October 6, 2020 at 1:19:26 PM UTC-6
>>>>>>>>>>>>> bell...@gmail.com wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> So, the [[[first/topic]]] are meant to be replaced with the
>>>>>>>>>>>>>> actual topic. So it would be something like this
>>>>>>>>>>>>>> ```
>>>>>>>>>>>>>> [[Topics]]
>>>>>>>>>>>>>> [[[topic name that snowDepth is published on]]]
>>>>>>>>>>>>>> [[[[topic name that snowDepth is published on]]]]
>>>>>>>>>>>>>> name = snowDepth
>>>>>>>>>>>>>> [[[topic name that snowRate is published on]]]
>>>>>>>>>>>>>> [[[[topic name that snowRate is published on]]]]
>>>>>>>>>>>>>> name = snowRate
>>>>>>>>>>>>>> ```
>>>>>>>>>>>>>> The duplication is an artifact of dealing with json and
>>>>>>>>>>>>>> keyword payloads. The ```use_topic_as_fieldname``` option
>>>>>>>>>>>>>> can be used to make the config a bit prettier.
>>>>>>>>>>>>>> ```
>>>>>>>>>>>>>> [[Topics]]
>>>>>>>>>>>>>> use_topic_as_fieldname = true
>>>>>>>>>>>>>> [[[topic name that snowDepth is published on]]]
>>>>>>>>>>>>>> name = snowDepth
>>>>>>>>>>>>>> [[[topic name that snowRate is published on]]]
>>>>>>>>>>>>>> name = snowRate
>>>>>>>>>>>>>> ```
>>>>>>>>>>>>>> Note, if snowDepth is actually published on the topic
>>>>>>>>>>>>>> snowDepth, the ```name``` option can be left off.
>>>>>>>>>>>>>> I don’t think that you want to set contains_total=true for
>>>>>>>>>>>>>> snowDepth. This is used when the field contains a total and it
>>>>>>>>>>>>>> needs to be
>>>>>>>>>>>>>> converted into an increment for WeeWX.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I’ll work on clarifying the wiki.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> With that said, it shouldn’t have broken WeeWX. If you are up
>>>>>>>>>>>>>> for it, before changing the config, setting debug=1, restarting
>>>>>>>>>>>>>> WeeWX for a
>>>>>>>>>>>>>> couple of archive intervals and attaching the log would be
>>>>>>>>>>>>>> appreciated .
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> rich
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tuesday, 6 October 2020 at 13:28:34 UTC-4
>>>>>>>>>>>>>> timothye...@gmail.com wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Attached is the extension material that I put into
>>>>>>>>>>>>>>> weewx.conf. But when I set enable = true, weewx stops archiving
>>>>>>>>>>>>>>> data. Is
>>>>>>>>>>>>>>> there an error in this configuration, or could subscribe be
>>>>>>>>>>>>>>> incompatible
>>>>>>>>>>>>>>> with another service? I'm using the Weatherflowudp driver with
>>>>>>>>>>>>>>> mqtt and
>>>>>>>>>>>>>>> GW1000 extensions.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tuesday, October 6, 2020 at 9:17:07 AM UTC-6 Timothy
>>>>>>>>>>>>>>> Buchanan wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks, Rich, I should be able to edit weewx.conf based on
>>>>>>>>>>>>>>>> the example at the bottom of that page.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I am using an ESP8266 board with an ultrasonic sensor and a
>>>>>>>>>>>>>>>> temperature sensor (to calibrate the speed of sound), and
>>>>>>>>>>>>>>>> programming in
>>>>>>>>>>>>>>>> the Arduino IDE. I'll 3D print a case and mount it above my
>>>>>>>>>>>>>>>> deck. The
>>>>>>>>>>>>>>>> materials cost about $15.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'd be happy to post the code here, under a new topic, once
>>>>>>>>>>>>>>>> I get it working.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Timothy
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tuesday, October 6, 2020 at 8:53:45 AM UTC-6
>>>>>>>>>>>>>>>> storm...@gmail.com wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> What type of sensor are you using for measuring snow depth?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/f1f286dd-9d27-4da8-821d-b015bc609aa6n%40googlegroups.com.
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: Caught
unrecoverable exception:
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: ****
MQTTSubscribe: use_topic_as_fieldname is mutually exclusive with
[[[[fieldname]]]] configuring
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: **** Traceback
(most recent call last):
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: **** File
"/usr/share/weewx/weewxd", line 148, in main
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: ****
engine = weewx.engine.StdEngine(config_dict)
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: **** File
"/usr/share/weewx/weewx/engine.py", line 75, in __init__
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: ****
self.loadServices(config_dict)
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: **** File
"/usr/share/weewx/weewx/engine.py", line 138, in loadServices
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: **** obj =
weeutil.weeutil.get_object(svc)(self,config_dict)
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: **** File
"/usr/share/weewx/user/MQTTSubscribe.py", line 1528, in __init__
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: ****
self.subscriber = MQTTSubscriber(service_dict, self.logger)
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: **** File
"/usr/share/weewx/user/MQTTSubscribe.py", line 1305, in __init__
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: ****
self.manager = TopicManager(topics_dict, self.logger)
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: **** File
"/usr/share/weewx/user/MQTTSubscribe.py", line 679, in __init__
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: **** raise
ValueError("MQTTSubscribe: use_topic_as_fieldname is mutually exclusive with
[[[[fieldname]]]] configuring")
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: ****
ValueError: MQTTSubscribe: use_topic_as_fieldname is mutually exclusive with
[[[[fieldname]]]] configuring
Oct 11 19:17:08 raspberrypi weewx[25990] CRITICAL __main__: **** Exiting.
# The topics to subscribe to.
[[topics]]
# Units for MQTT payloads without unit value.
# Valid values: US, METRIC, METRICWX
# Default is: US
unit_system = US
ignore_start_time = true
ignore_end_time = true
use_topic_as_fieldname = true
# The first topic
# MQTT Topic
[[[snow/snowDepth]]]
# MQTT name
[[[[snowDepth]]]]
# weewx name
name = snowDepth
ignore = false
contains_total = false
conversion_type = float
units = inch
