Agreed, two separate problems. Not yet sure about the units problem. But, you don’t need that option. I opened an issue to dig into it when I have time. The second is because it is an invalid configuration. Have you tried the snippet that is in my last post? Namely, [[topics]] # Units for MQTT payloads without unit value. # Valid values: US, METRIC, METRICWX # Default is: US unit_system = US ignore_start_time = true ignore_end_time = true use_topic_as_fieldname = true # The first topic [[[snowDepth]]]
Or with nested topics, [[topics]] # Units for MQTT payloads without unit value. # Valid values: US, METRIC, METRICWX # Default is: US unit_system = US ignore_start_time = true ignore_end_time = true use_topic_as_fieldname = true # The first topic [[[snow/snowDepth]]] name = snowDepth These would be the simplest to start with. And again, full logs instead of snippets would make this so much easier. rich On Monday, 12 October 2020 at 12:57:25 UTC-4 timothye...@gmail.com wrote: > See attached. > > On Monday, October 12, 2020 at 9:36:08 AM UTC-6 bell...@gmail.com wrote: > >> >> Well, I don’t understand how that configuration even got that far. I >> would have expected the previous error. But just seeing log snippets is >> like walking around blindfolded. Base on your last config snippet, replace >> it with >> >> The topics to subscribe to. >> [[topics]] >> # Units for MQTT payloads without unit value. >> # Valid values: US, METRIC, METRICWX >> # Default is: US >> unit_system = US >> ignore_start_time = true >> ignore_end_time = true >> use_topic_as_fieldname = true >> >> # The first topic >> # MQTT Topic >> [[[snowDepth]]] >> >> This last error has to do with trying to convert the units of snowDepth. >> Either it WeeWX does not fully support that, or more likely I am doing >> something wrong. But since everything is in the US unit_system, you don’t >> need to specify units at the field level. >> >> On Monday, 12 October 2020 at 10:38:14 UTC-4 timothye...@gmail.com wrote: >> >>> I decided to try simplifying the topic to just snowDepth, as this: >>> >>> The topics to subscribe to. >>> [[topics]] >>> # Units for MQTT payloads without unit value. >>> # Valid values: US, METRIC, METRICWX >>> # Default is: US >>> unit_system = US >>> ignore_start_time = true >>> ignore_end_time = true >>> use_topic_as_fieldname = true >>> >>> # The first topic >>> # MQTT Topic >>> [[[snowDepth]]] >>> # MQTT name >>> [[[[snowDepth]]]] >>> # weewx name >>> name = snowDepth >>> ignore = false >>> contains_total = false >>> conversion_type = float >>> units = inch >>> >>> when I publish 2.5 to snowDepth, I get this error in syslog: >>> >>> Oct 12 08:12:52 raspberrypi weewx[29222] DEBUG user.MQTTSubscribe: >>> (Service) MessageCallbackProvider data-> incoming topic: snowDepth, QOS: 0, >>> retain: 0, payload: b'2.5' >>> Oct 12 08:12:52 raspberrypi weewx[29222] DEBUG weewx.units: Unable to >>> convert from inch to None >>> Oct 12 08:12:52 raspberrypi weewx[29222] ERROR user.MQTTSubscribe: >>> (Service) MessageCallbackProvider on_message_individual failed with: None >>> Oct 12 08:12:52 raspberrypi weewx[29222] ERROR user.MQTTSubscribe: >>> (Service) **** MessageCallbackProvider Ignoring topic=snowDepth and >>> payload=b'2.5' >>> >>> Could you perhaps post a working configuration file, any topic, any >>> unit, etc. just one that works on your weewx and I will plug it in and see >>> what happens? >>> >>> On Sunday, October 11, 2020 at 7:27:15 PM UTC-6 Timothy Buchanan wrote: >>> >>>> See attached. >>>> >>>> On Sunday, October 11, 2020 at 5:45:43 PM UTC-6 bell...@gmail.com >>>> wrote: >>>> >>>>> Sorry, I didn’t get the config quite right. I don’t subscribe to >>>>> ‘individual’ topics, so this is untested, but should work. If it >>>>> doesn’t, >>>>> I’ll spin up a debug environment to figure it out. >>>>> [[topics]] >>>>> Keep the current settings, unit_system, ignore_start_time, >>>>> ignore_end_time, etc. >>>>> >>>>> # The next option will take the tail end of the topic and use it as >>>>> the MQTT field name >>>>> # So, if the topic is snow/snowDepth, the MQTT field name is >>>>> snowDepth >>>>> # Only valid when the payload is type ‘individual’ >>>>> use_topic_as_fieldname = true >>>>> [[[snow/snowDepth]]] >>>>> # Because snowDepth is the WeeWX name and the defaults work for >>>>> you, nothing else is needed. >>>>> [[[snow/snowRate]]] >>>>> # Not sure, but the defaults should probably work here... >>>>> >>>>> Again, if this doesn’t work; post the log and I’ll spin up a debug >>>>> environment to figure out what is going on. >>>>> rich >>>>> On Sunday, 11 October 2020 at 16:43:58 UTC-4 timothye...@gmail.com >>>>> wrote: >>>>> >>>>>> After moving the ignore lines, I am happy to report that the time >>>>>> problem is solved; that is, it is no longer rejecting the topic. >>>>>> However, >>>>>> it is still not making it into the database. I think this is because it >>>>>> is >>>>>> not converting from snow/snowDepth to snowDepth. See attached syslog >>>>>> extract and weewx.conf for section where topic is configured. Is there a >>>>>> mistake in this topic configuration? >>>>>> >>>>>> On Sunday, October 11, 2020 at 11:26:27 AM UTC-6 bell...@gmail.com >>>>>> wrote: >>>>>> >>>>>>> I've run both extensions off and on, so I don't think there is any >>>>>>> conflict. The root-root cause is time skew between your pi and weather >>>>>>> station and MQTTSubscribe being a bit overly aggressive on its quality >>>>>>> control. Setting ignore_start_time and ignore_end_time should turn this >>>>>>> off >>>>>>> completely. Since the incoming data has no dateTime, this seems the >>>>>>> most >>>>>>> logical approach. >>>>>>> >>>>>>> I noticed in the latest attached config, those options are in the >>>>>>> wrong place. Move them under [[topics]] after >>>>>>> unit_system = US >>>>>>> ignore_start_time = true >>>>>>> ignore_end_time = true >>>>>>> Also, Ignore_end_time has a capital "I". >>>>>>> If it is still not working, please attach the log from startup. That >>>>>>> way we can see how the options are being processed. >>>>>>> I think we are real close. -rich >>>>>>> >>>>>>> On Sunday, 11 October 2020 at 12:20:32 UTC-4 timothye...@gmail.com >>>>>>> wrote: >>>>>>> >>>>>>>> I used wee_extension to uninstall the old version then install the >>>>>>>> new. I used the configuration as shown in the attached file. When I >>>>>>>> publish >>>>>>>> to a topic, Topic Manager still ignores it as outside of interval >>>>>>>> (syslog >>>>>>>> extract in attached file). >>>>>>>> >>>>>>>> I'm thinking that MQTTSubscribe is not compatible with the mqtt >>>>>>>> extension. I note that MQTTSubscribe is publishing records of data >>>>>>>> that it >>>>>>>> is not subscribed to, but which are published by mqtt. Have you tried >>>>>>>> running MQTTSubscribe and mqtt together? >>>>>>>> >>>>>>>> On Saturday, October 10, 2020 at 5:10:34 PM UTC-6 bell...@gmail.com >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I created a pre-release with the fix. It is here, >>>>>>>>> https://github.com/bellrichm/WeeWX-MQTTSubscribe/releases/tag/v1.6.2-rc03 >>>>>>>>> Hopefully this is it. >>>>>>>>> rich >>>>>>>>> On Saturday, 10 October 2020 at 10:15:29 UTC-4 >>>>>>>>> timothye...@gmail.com wrote: >>>>>>>>> >>>>>>>>>> OK, thanks for your help. >>>>>>>>>> >>>>>>>>>> On Saturday, October 10, 2020 at 7:22:18 AM UTC-6 >>>>>>>>>> bell...@gmail.com wrote: >>>>>>>>>> >>>>>>>>>>> Thanks for the log. Looks like there is a bug with the >>>>>>>>>>> ignore_start_time option. Fix looks easy, I just need to make sure >>>>>>>>>>> there >>>>>>>>>>> are no side effects. >>>>>>>>>>> I’ll let you know when the fix is available. >>>>>>>>>>> - rich >>>>>>>>>>> >>>>>>>>>>> On Friday, 9 October 2020 at 22:55:11 UTC-4 >>>>>>>>>>> timothye...@gmail.com wrote: >>>>>>>>>>> >>>>>>>>>>>> I added those this way: >>>>>>>>>>>> >>>>>>>>>>>> # The topics to subscribe to. >>>>>>>>>>>> [[topics]] >>>>>>>>>>>> # Units for MQTT payloads without unit value. >>>>>>>>>>>> # Valid values: US, METRIC, METRICWX >>>>>>>>>>>> # Default is: US >>>>>>>>>>>> unit_system = US >>>>>>>>>>>> ignore_start_time = true >>>>>>>>>>>> Ignore_end_time = true >>>>>>>>>>>> >>>>>>>>>>>> but I still get this: >>>>>>>>>>>> >>>>>>>>>>>> Oct 9 20:38:29 raspberrypi weewx[30461] INFO weewx.restx: >>>>>>>>>>>> MQTT: Published record 2020-10-09 20:38:47 MDT (1602297527) >>>>>>>>>>>> Oct 9 20:38:30 raspberrypi weewx[30461] DEBUG >>>>>>>>>>>> user.MQTTSubscribe: (Service) MessageCallbackProvider data-> >>>>>>>>>>>> incoming >>>>>>>>>>>> topic: snow/snowDepth, QOS: 0, retain: 0, payload: b'7' >>>>>>>>>>>> Oct 9 20:38:30 raspberrypi weewx[30461] DEBUG >>>>>>>>>>>> user.MQTTSubscribe: (Service) TopicManager data-> incoming >>>>>>>>>>>> snow/snowDepth: >>>>>>>>>>>> snow/snowDepth: 7.0 >>>>>>>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] DEBUG >>>>>>>>>>>> user.MQTTSubscribe: (Service) TopicManager data-> outgoing >>>>>>>>>>>> snow/snowDepth: >>>>>>>>>>>> dateTime: 1602297510.2596123, snow/snowDepth: 7.0, usUnits: 1 >>>>>>>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] INFO >>>>>>>>>>>> user.MQTTSubscribe: (Service) TopicManager ignoring record outside >>>>>>>>>>>> of >>>>>>>>>>>> interval 1602297510.259612 1602297529.000000 1602297510.259612 >>>>>>>>>>>> dateTime: >>>>>>>>>>>> 1602297510.2596123, snow/snowDepth: 7.0, usUnits: 1 >>>>>>>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] DEBUG >>>>>>>>>>>> user.MQTTSubscribe: (Service) TopicManager data-> outgoing >>>>>>>>>>>> accumulated >>>>>>>>>>>> snow/snowDepth: >>>>>>>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] DEBUG >>>>>>>>>>>> user.MQTTSubscribe: (Service) data-> final packet is 2020-10-09 >>>>>>>>>>>> 20:38:49 >>>>>>>>>>>> MDT (1602297529): avg_distance: 0, dateTime: 1602297529, >>>>>>>>>>>> lightning_strikes: >>>>>>>>>>>> 0, outHumidity: 23, outTemp: 13.34, outTempBatteryStatus: 2.919, >>>>>>>>>>>> pressure: >>>>>>>>>>>> 744.7, usUnits: 17 >>>>>>>>>>>> Oct 9 20:38:31 raspberrypi weewx[30461] INFO weewx.restx: >>>>>>>>>>>> MQTT: Published record 2020-10-09 20:38:49 MDT (1602297529) >>>>>>>>>>>> Oct 9 20:38:32 raspberrypi weewx[30461] DEBUG >>>>>>>>>>>> user.MQTTSubscribe: (Service) data-> final packet is 2020-10-09 >>>>>>>>>>>> 20:38:50 >>>>>>>>>>>> MDT (1602297530): dateTime: 1602297530, usUnits: 17, windDir: 336, >>>>>>>>>>>> windSpeed: 0.72 >>>>>>>>>>>> Oct 9 20:38:32 raspberrypi weewx[30461] INFO weewx.restx: >>>>>>>>>>>> MQTT: Published record 2020-10-09 20:38:50 MDT (1602297530) >>>>>>>>>>>> >>>>>>>>>>>> On Friday, October 9, 2020 at 5:13:41 PM UTC-6 >>>>>>>>>>>> bell...@gmail.com wrote: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Ah, crap. This has to do with an attempt to quality control >>>>>>>>>>>>> data by time. Next major release, I really need to rework it... >>>>>>>>>>>>> Since you have individual payloads (no timestamp from the >>>>>>>>>>>>> origin) add the following under [[topics]] >>>>>>>>>>>>> ignore_start_time = true >>>>>>>>>>>>> Ignore_end_time = true >>>>>>>>>>>>> >>>>>>>>>>>>> We are getting close. Sorry for the pain. >>>>>>>>>>>>> rich >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Friday, 9 October 2020 at 18:36:27 UTC-4 >>>>>>>>>>>>> timothye...@gmail.com wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> syslog extract posted. weewx is loading the service, though >>>>>>>>>>>>>> it also seems to subscribe to the data which MQTT is publishing >>>>>>>>>>>>>> from weewx. >>>>>>>>>>>>>> the second section shows that subscribe receives the data I >>>>>>>>>>>>>> published from >>>>>>>>>>>>>> the terminal (6 inches of snow), but rejects it as outside of >>>>>>>>>>>>>> interval. Is >>>>>>>>>>>>>> there an entry to be made somewhere to sync incoming data? I >>>>>>>>>>>>>> note that >>>>>>>>>>>>>> another extension I use (GW1000) needed dateTime = datetime in a >>>>>>>>>>>>>> field map >>>>>>>>>>>>>> to work properly. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Friday, October 9, 2020 at 2:54:56 PM UTC-6 >>>>>>>>>>>>>> bell...@gmail.com wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> That could be simplified, but looks like it should work. The >>>>>>>>>>>>>>> quickest and easiest way to proceed is to set debug to 1, >>>>>>>>>>>>>>> restart WeeWX, >>>>>>>>>>>>>>> let it run through an archive cycle during which you published >>>>>>>>>>>>>>> to that >>>>>>>>>>>>>>> topic and then post the log. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Re: snowBatteryStatus - The units config option is only >>>>>>>>>>>>>>> needed if the field units do not match the units expected by >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> unit_system. So eliminating will at least get the data in the >>>>>>>>>>>>>>> DB. >>>>>>>>>>>>>>> Note, units is not needed for snowDepth because inches is >>>>>>>>>>>>>>> the Units for that field in the US unit_system. >>>>>>>>>>>>>>> rich >>>>>>>>>>>>>>> On Friday, 9 October 2020 at 14:23:01 UTC-4 >>>>>>>>>>>>>>> timothye...@gmail.com wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I've attached a syslog extract showing where weewx crashed. >>>>>>>>>>>>>>>> it seems that "volt" is an invalid unit for my topic. i don't >>>>>>>>>>>>>>>> know why but >>>>>>>>>>>>>>>> for now I commented out that topic and its parameters. Now, >>>>>>>>>>>>>>>> weewx will >>>>>>>>>>>>>>>> continue running when subscribe is enabled, but subscribed >>>>>>>>>>>>>>>> topics are not >>>>>>>>>>>>>>>> being posted to the database. Here is the first topic in >>>>>>>>>>>>>>>> weewx.conf: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> # The first topic >>>>>>>>>>>>>>>> # MQTT Topic >>>>>>>>>>>>>>>> [[[snow/snowDepth]]] >>>>>>>>>>>>>>>> # MQTT name >>>>>>>>>>>>>>>> [[[[snowDepth]]]] >>>>>>>>>>>>>>>> # weewx name >>>>>>>>>>>>>>>> name = snowDepth >>>>>>>>>>>>>>>> ignore = false >>>>>>>>>>>>>>>> contains_total = false >>>>>>>>>>>>>>>> conversion_type = float >>>>>>>>>>>>>>>> units = inch >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I used a terminal to publish "6" to snow/snowDepth on >>>>>>>>>>>>>>>> Mosquitto. Another terminal window command to subscribe to >>>>>>>>>>>>>>>> snow/snowDepth >>>>>>>>>>>>>>>> received the "6" but the database entries for snowDepth are >>>>>>>>>>>>>>>> null. Is this >>>>>>>>>>>>>>>> configuration of topics still not correct. Thanks. >>>>>>>>>>>>>>>> On Tuesday, October 6, 2020 at 1:43:31 PM UTC-6 Timothy >>>>>>>>>>>>>>>> Buchanan wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, Rich. I'll try it when I get back. I'll be in our >>>>>>>>>>>>>>>>> second home for a few days: going to the gun range and the >>>>>>>>>>>>>>>>> clothing-optional resort (not at the same time!). >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Tuesday, October 6, 2020 at 1:19:26 PM UTC-6 >>>>>>>>>>>>>>>>> bell...@gmail.com wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> So, the [[[first/topic]]] are meant to be replaced with >>>>>>>>>>>>>>>>>> the actual topic. So it would be something like this >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> [[Topics]] >>>>>>>>>>>>>>>>>> [[[topic name that snowDepth is published on]]] >>>>>>>>>>>>>>>>>> [[[[topic name that snowDepth is published on]]]] >>>>>>>>>>>>>>>>>> name = snowDepth >>>>>>>>>>>>>>>>>> [[[topic name that snowRate is published on]]] >>>>>>>>>>>>>>>>>> [[[[topic name that snowRate is published on]]]] >>>>>>>>>>>>>>>>>> name = snowRate >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> The duplication is an artifact of dealing with json and >>>>>>>>>>>>>>>>>> keyword payloads. The ```use_topic_as_fieldname``` >>>>>>>>>>>>>>>>>> option can be used to make the config a bit prettier. >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> [[Topics]] >>>>>>>>>>>>>>>>>> use_topic_as_fieldname = true >>>>>>>>>>>>>>>>>> [[[topic name that snowDepth is published on]]] >>>>>>>>>>>>>>>>>> name = snowDepth >>>>>>>>>>>>>>>>>> [[[topic name that snowRate is published on]]] >>>>>>>>>>>>>>>>>> name = snowRate >>>>>>>>>>>>>>>>>> ``` >>>>>>>>>>>>>>>>>> Note, if snowDepth is actually published on the topic >>>>>>>>>>>>>>>>>> snowDepth, the ```name``` option can be left off. >>>>>>>>>>>>>>>>>> I don’t think that you want to set contains_total=true >>>>>>>>>>>>>>>>>> for snowDepth. This is used when the field contains a total >>>>>>>>>>>>>>>>>> and it needs to >>>>>>>>>>>>>>>>>> be converted into an increment for WeeWX. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I’ll work on clarifying the wiki. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> With that said, it shouldn’t have broken WeeWX. If you >>>>>>>>>>>>>>>>>> are up for it, before changing the config, setting debug=1, >>>>>>>>>>>>>>>>>> restarting >>>>>>>>>>>>>>>>>> WeeWX for a couple of archive intervals and attaching the >>>>>>>>>>>>>>>>>> log would be >>>>>>>>>>>>>>>>>> appreciated . >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> rich >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Tuesday, 6 October 2020 at 13:28:34 UTC-4 >>>>>>>>>>>>>>>>>> timothye...@gmail.com wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Attached is the extension material that I put into >>>>>>>>>>>>>>>>>>> weewx.conf. But when I set enable = true, weewx stops >>>>>>>>>>>>>>>>>>> archiving data. Is >>>>>>>>>>>>>>>>>>> there an error in this configuration, or could subscribe be >>>>>>>>>>>>>>>>>>> incompatible >>>>>>>>>>>>>>>>>>> with another service? I'm using the Weatherflowudp driver >>>>>>>>>>>>>>>>>>> with mqtt and >>>>>>>>>>>>>>>>>>> GW1000 extensions. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Tuesday, October 6, 2020 at 9:17:07 AM UTC-6 Timothy >>>>>>>>>>>>>>>>>>> Buchanan wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, Rich, I should be able to edit weewx.conf based >>>>>>>>>>>>>>>>>>>> on the example at the bottom of that page. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I am using an ESP8266 board with an ultrasonic sensor >>>>>>>>>>>>>>>>>>>> and a temperature sensor (to calibrate the speed of >>>>>>>>>>>>>>>>>>>> sound), and programming >>>>>>>>>>>>>>>>>>>> in the Arduino IDE. I'll 3D print a case and mount it >>>>>>>>>>>>>>>>>>>> above my deck. The >>>>>>>>>>>>>>>>>>>> materials cost about $15. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I'd be happy to post the code here, under a new topic, >>>>>>>>>>>>>>>>>>>> once I get it working. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Timothy >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Tuesday, October 6, 2020 at 8:53:45 AM UTC-6 >>>>>>>>>>>>>>>>>>>> storm...@gmail.com wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> What type of sensor are you using for measuring snow >>>>>>>>>>>>>>>>>>>>> depth? >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/308b00c0-b404-4190-9a88-f739ab5a9738n%40googlegroups.com.