Rich with your suggestion, barometer now works, thanks, Gene On Saturday, August 14, 2021 at 1:39:00 PM UTC-4 [email protected] wrote:
> In the log should be more info with the error message. I think it might be > multiple lines, which is why it did not show up in the grep. > > But, I think the problem is that we don’t have the ‘type’ setting in quite > the correct place. Try something like this > > > [[topics]] > > [[[message]]] > > type = json > > keyword_separator = ":" > > keyword_delimiter = "," > > # Units for MQTT payloads without unit value. > > # Valid values: US, METRIC, METRICWX > > # For more information see, > http://weewx.com/docs/customizing.htm#units > > # Default is US > > # unit_system = METRIC > > [[[homeassistant/sensor/barometer]]] > > [[[[message]]]] > > type = individual > > [[[[homeassistant/sensor/barometer]]]] > > name = barometer > > units = mbar > > > Sorry for the tenseness of the replies, I’m away from my computer for a > few days. > > rich > > On Saturday, 14 August 2021 at 11:53:05 UTC-4 [email protected] wrote: > >> Rich, I set up for both json and individual mqtt callback types, the >> beginning of my topics section in [MQTTSubscribeDriver] is shown, the >> remainder of the topic section is the same as my previous post. I am unable >> to get the first topic (homeassistant/sensor/barometer) running as >> individual callback to be recorded in weewx. I ran in logging level = >> DEBUG and looking grep barometer mqtt.log show at bottom, I see that the >> barometer topic is set up, read, with a value of 1025, but the last line >> shows that its value is ignored. Any thoughts why not working? >> >> # [[message_callback]] >> # The format of the MQTT payload. >> # Currently support: individual, json, keyword >> # Must be specified. >> [[topics]] >> [[[message]]] >> type = json >> keyword_separator = ":" >> keyword_delimiter = "," >> # Units for MQTT payloads without unit value. >> # Valid values: US, METRIC, METRICWX >> # For more information see, >> http://weewx.com/docs/customizing.htm#units >> # Default is US >> # unit_system = METRIC >> [[[homeassistant/sensor/barometer]]] >> [[[[homeassistant/sensor/barometer]]]] >> type = individual >> name = barometer >> units = mbar >> >> [[[rtl_433/9b13b3f4-rtl433/events]]] >> msg_id_field = id >> [[[[time]]]] >> ignore_msg_id_field = True >> ignore = True >> [[[[model]]]] >> ignore_msg_id_field = True >> ignore = True >> >> ......... >> >> >> grep barometer mqtt.log >> 2021-08-14 11:07:51 DEBUG (Driver) MQTTSUBscriber sanitized_service_dict >> is {u'username': u'----', u'log': u'true', u'topics': {u'message': >> {u'type': u'json', u'keyword_separator': u':', u'keyword_delimiter': u','}, >> u'homeassistant/sensor/barometer': {u'homeassistant/sensor/barometer': >> {u'type': u'individual', u'name': u'barometer', u'units': u'mbar', >> u'ignore': u'False'}}, u'rtl_433/9b13b3f4-rtl433/events': {u'msg_id_field': >> u'id', u'time': {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, >> u'model': {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'id': >> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'channel': >> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'battery_ok': >> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, >> u'temperature_C_223': {u'name': u'outTemp', u'units': u'degree_C'}, >> u'humidity_223': {u'name': u'outHumidity'}, u'strike_count_223': {u'name': >> u'lightning_strike_count', u'contains_total': u'True'}, u'storm_dist_223': >> {u'name': u'lightning_distance', u'units': u'mile'}, u'active': >> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'rfi': >> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'exception': >> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'raw_msg': >> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'mic': >> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'rain_mm_1322': >> {u'name': u'rain', u'contains_total': u'true', u'units': u'mm'}}}, >> u'driver': u'user.MQTTSubscribe', u'logging_level': u'DEBUG', u'host': >> u'localhost', u'keepalive': u'60', u'port': u'1883', u'logging_filename': >> u'/home/pi/mqtt.log'} >> 2021-08-14 11:07:51 DEBUG (Driver) TopicManager self.subscribed_topics is >> {"homeassistant/sensor/barometer": {"datetime_format": null, >> "msg_id_field": null, "qos": 0, "filters": {}, "fields": >> {"homeassistant/sensor/barometer": {"name": "barometer", "contains_total": >> false, "conversion_error_to_none": false, "ignore": false, >> "conversion_func": {"source": "lambda x: to_float(x)", "compiled": >> "<function <lambda> at 0x761368b0>"}, "units": "mbar"}}, >> "message-1628953671.535918": {"type": "json", "keyword_separator": ":", >> "keyword_delimiter": ","}, "use_server_datetime": false, "queue": {"name": >> "homeassistant/sensor/barometer", "type": "normal", "adjust_start_time": >> 0.0, "adjust_end_time": 0.0, "ignore_start_time": false, "ignore_end_time": >> false, "data": "deque([])", "max_size": 2147483647}, "ignore_msg_id_field": >> [], "ignore": false, "subscribe": true, "conversion_func": {"source": >> "lambda x: to_float(x)", "compiled": "<function <lambda> at 0x76200970>"}, >> "unit_system": 1, "topic_tail_is_fieldname": false, "offset_format": null}, >> "rtl_433/9b13b3f4-rtl433/events": {"datetime_format": null, "msg_id_field": >> "id", "qos": 0, "filters": {}, "fields": {"exception": {"ignore": true, >> "conversion_func": {"source": "lambda x: to_float(x)", "compiled": >> "<function <lambda> at 0x7614ecb0>"}, "contains_total": false, "name": >> "exception", "conversion_error_to_none": false}, "temperature_C_223": >> {"name": "outTemp", "contains_total": false, "conversion_error_to_none": >> false, "ignore": false, "conversion_func": {"source": "lambda x: >> to_float(x)", "compiled": "<function <lambda> at 0x761366b0>"}, "units": >> "degree_C"}, "rain_mm_1322": {"name": "rain", "contains_total": true, >> "conversion_error_to_none": false, "ignore": false, "conversion_func": >> {"source": "lambda x: to_float(x)", "compiled": "<function <lambda> at >> 0x761530f0>"}, "units": "mm"}, "strike_count_223": {"ignore": false, >> "conversion_func": {"source": "lambda x: to_float(x)", "compiled": >> "<function <lambda> at 0x7614ad70>"}, "contains_total": true, "name": >> "lightning_strike_count", "conversion_error_to_none": false}, "battery_ok": >> {"ignore": true, "conversion_func": {"source": "lambda x: to_float(x)", >> "compiled": "<function <lambda> at 0x76136bb0>"}, "contains_total": false, >> "name": "battery_ok", "conversion_error_to_none": false}, "humidity_223": >> {"ignore": false, "conversion_func": {"source": "lambda x: to_float(x)", >> "compiled": "<function <lambda> at 0x7614a9b0>"}, "contains_total": false, >> "name": "outHumidity", "conversion_error_to_none": false}, "mic": >> {"ignore": true, "conversion_func": {"source": "lambda x: to_float(x)", >> "compiled": "<function <lambda> at 0x7614e3f0>"}, "contains_total": false, >> "name": "mic", "conversion_error_to_none": false}, "raw_msg": {"ignore": >> true, "conversion_func": {"source": "lambda x: to_float(x)", "compiled": >> "<function <lambda> at 0x7614ebb0>"}, "contains_total": false, "name": >> "raw_msg", "conversion_error_to_none": false}, "rfi": {"ignore": true, >> "conversion_func": {"source": "lambda x: to_float(x)", "compiled": >> "<function <lambda> at 0x7614eb70>"}, "contains_total": false, "name": >> "rfi", "conversion_error_to_none": false}, "storm_dist_223": {"name": >> "lightning_distance", "contains_total": false, "conversion_error_to_none": >> false, "ignore": false, "conversion_func": {"source": "lambda x: >> to_float(x)", "compiled": "<function <lambda> at 0x7614a770>"}, "units": >> "mile"}, "time": {"ignore": true, "conversion_func": {"source": "lambda x: >> to_float(x)", "compiled": "<function <lambda> at 0x761367f0>"}, >> "contains_total": false, "name": "time", "conversion_error_to_none": >> false}, "active": {"ignore": true, "conversion_func": {"source": "lambda x: >> to_float(x)", "compiled": "<function <lambda> at 0x7614e730>"}, >> "contains_total": false, "name": "active", "conversion_error_to_none": >> false}, "model": {"ignore": true, "conversion_func": {"source": "lambda x: >> to_float(x)", "compiled": "<function <lambda> at 0x761367b0>"}, >> "contains_total": false, "name": "model", "conversion_error_to_none": >> false}, "id": {"ignore": true, "conversion_func": {"source": "lambda x: >> to_float(x)", "compiled": "<function <lambda> at 0x76136730>"}, >> "contains_total": false, "name": "id", "conversion_error_to_none": false}, >> "channel": {"ignore": true, "conversion_func": {"source": "lambda x: >> to_float(x)", "compiled": "<function <lambda> at 0x76136370>"}, >> "contains_total": false, "name": "channel", "conversion_error_to_none": >> false}}, "message-1628953671.535918": {"type": "json", "keyword_separator": >> ":", "keyword_delimiter": ","}, "use_server_datetime": false, "queue": >> {"name": "rtl_433/9b13b3f4-rtl433/events", "type": "normal", >> "adjust_start_time": 0.0, "adjust_end_time": 0.0, "ignore_start_time": >> false, "ignore_end_time": false, "data": "deque([])", "max_size": >> 2147483647}, "ignore_msg_id_field": ["time", "model", "id", "channel", >> "battery_ok", "active", "rfi", "exception", "raw_msg", "mic"], "ignore": >> false, "subscribe": true, "conversion_func": {"source": "lambda x: >> to_float(x)", "compiled": "<function <lambda> at 0x761365b0>"}, >> "unit_system": 1, "topic_tail_is_fieldname": false, "offset_format": null}, >> "1628953671.557307-windGust-windGustDir-windDir-windSpeed": >> {"datetime_format": null, "queue": {"name": >> "1628953671.557307-windGust-windGustDir-windDir-windSpeed", "type": >> "collector", "adjust_start_time": 0.0, "adjust_end_time": 0.0, >> "ignore_start_time": false, "ignore_end_time": false, "data": "deque([])", >> "max_size": 2147483647}, "qos": 0, "message-1628953671.535918": {}, >> "use_server_datetime": false, "offset_format": null, "subscribe": false, >> "unit_system": 1, "topic_tail_is_fieldname": false}} >> 2021-08-14 11:07:51 DEBUG (Driver) MQTTSubscribe MQTT: Sending SUBSCRIBE >> (d0, m1) [('homeassistant/sensor/barometer', 0)] >> 2021-08-14 11:07:51 INFO (Driver) Subscribing to >> homeassistant/sensor/barometer has a mid 1 and rc 0 >> 2021-08-14 11:10:02 DEBUG (Driver) MQTTSubscribe MQTT: Received PUBLISH >> (d0, q0, r0, m0), 'homeassistant/sensor/barometer', ... (8 bytes) >> 2021-08-14 11:10:02 DEBUG (Driver) MessageCallbackProvider data-> >> incoming topic: homeassistant/sensor/barometer, QOS: 0, retain: 0, payload: >> 1025.455 >> 2021-08-14 11:10:02 ERROR (Driver) **** MessageCallbackProvider Ignoring >> topic=homeassistant/sensor/barometer and payload=1025.455 >> >> On Friday, August 13, 2021 at 9:53:04 PM UTC-4 [email protected] wrote: >> >>> Good to hear you have something working. >>> Here is some information on configuring the message type by payload. >>> >>> https://github.com/bellrichm/WeeWX-MQTTSubscribe/issues/113#issuecomment-809655246 >>> This exists on the version you have. If you have any feedback, let me >>> know. >>> rich >>> >>> >>> On Friday, 13 August 2021 at 17:03:28 UTC-4 [email protected] wrote: >>> >>>> Rich, I have solved problem for persisting distance values without >>>> lightning strikes, I am now using json rather than individual message >>>> callback, this leads to both lightning_strikes and lightning_distance to >>>> be >>>> in same loop which as you predicted would solved the problem. My >>>> [MQTTSubscribeDriver] entry for the acurite 6054 lightning detector and >>>> acurite -rain899 rainfall gauge are included below, my example shows >>>> working for msg_id_field for acurite 6054 to be 233 and the acurite >>>> -rain899 to be 1322. Msg_id_field may differ for other peoples devices, >>>> but I don't know. >>>> >>>> One question remains, as usual when you solve one problem, you create >>>> another, I have sensors which for their mqtt output are not formatted in >>>> json, but work using "individual" callback. Some might be convertible to >>>> json but others will not. I see you have in configuring experimental >>>> options, the option to pick for each topic the type of format, with "name >>>> = >>>> value format" but I am not sure where that goes or if it will work with my >>>> current version (2.1.0rc). Thanks for the suggestions and the great work >>>> on MQTTSubscribeDriver. Gene >>>> >>>> >>>> >>>> >>>> [[message_callback]] >>>> # The format of the MQTT payload. >>>> # Currently support: individual, json, keyword >>>> # Must be specified. >>>> type = json >>>> # keyword_separator = ":" >>>> # keyword_delimiter = "," >>>> [[topics]] >>>> # Units for MQTT payloads without unit value. >>>> # Valid values: US, METRIC, METRICWX >>>> # For more information see, >>>> http://weewx.com/docs/customizing.htm#units >>>> # Default is US >>>> unit_system = METRIC >>>> [[[rtl_433/9b13b3f4-rtl433/events]]] >>>> msg_id_field = id >>>> [[[[time]]]] >>>> ignore_msg_id_field = True >>>> ignore = True >>>> [[[[model]]]] >>>> ignore_msg_id_field = True >>>> ignore = True >>>> [[[[id]]]] >>>> ignore_msg_id_field = True >>>> ignore = True >>>> [[[[channel]]]] >>>> ignore_msg_id_field = True >>>> ignore = True >>>> [[[[battery_ok]]]] >>>> ignore_msg_id_field = True >>>> ignore = True >>>> [[[[temperature_C_223]]]] >>>> name = outTemp >>>> units = degree_C >>>> [[[[humidity_223]]]] >>>> name = outHumidity >>>> [[[[strike_count_223]]]] >>>> name = lightning_strike_count >>>> contains_total = True >>>> [[[[storm_dist_223]]]] >>>> name = lightning_distance >>>> units = mile >>>> [[[[active]]]] >>>> ignore_msg_id_field = True >>>> ignore = True >>>> [[[[rfi]]]] >>>> ignore_msg_id_field = True >>>> ignore = True >>>> [[[[exception]]]] >>>> ignore_msg_id_field = True >>>> ignore = True >>>> [[[[raw_msg]]]] >>>> ignore_msg_id_field = True >>>> ignore = True >>>> [[[[mic]]]] >>>> ignore_msg_id_field = True >>>> ignore = True >>>> [[[[rain_mm_1322]]]] >>>> name = rain >>>> contains_total = true >>>> units = mm >>>> >>>> >>>> On Thursday, August 12, 2021 at 7:49:04 PM UTC-4 [email protected] >>>> wrote: >>>> >>>>> I am guessing that you are getting loop packets with just distance >>>>> because of the order the data is arriving over MQTT. >>>>> >>>>> For example: >>>>> message with distance arrives >>>>> - data is saved >>>>> another message with distance arrives >>>>> - packet with saved data is created >>>>> - current message is saved >>>>> message with count arrives >>>>> - count is saved >>>>> a message with either distance or count arrives >>>>> - packet with saved data is created >>>>> - data received is saved >>>>> >>>>> A debug level log would confirm this. >>>>> >>>>> If a ‘corrections’ expression won’t fix this, I will need a debug >>>>> level log to get an understanding of the data being received. >>>>> >>>>> If you are using rtl_433 to publish to MQTT, I’d also be interested in >>>>> what is being published to the ‘events’ topic. >>>>> rich >>>>> >>>>> On Thursday, 12 August 2021 at 16:45:35 UTC-4 [email protected] wrote: >>>>> >>>>>> Not quite bingo, I am still getting some fixed lightning_distance >>>>>> integers in the absence of strikes, about 6 per hour. Loops are >>>>>> produced >>>>>> with lightning_distance = 7.999 and no information on >>>>>> lightning_strike_count and other loops have with both lightning_distance >>>>>> = >>>>>> 7.999 and lightning_strike_count = 0 See below output from weewxd for >>>>>> 2 >>>>>> hours. >>>>>> >>>>>> I changed [[Corrections]] to >>>>>> lightning_distance = None if lightning_strike_count == 0 or None >>>>>> else lightning_distance >>>>>> >>>>>> thinking that this correction would set lightning_distance to none in >>>>>> the absence of a lightning strike count, but the correction had no >>>>>> effect. >>>>>> Looking at all the loops there is no reason some loops have >>>>>> lightning_distance: 7.99 while the great majority have the desired >>>>>> lightning_distance: None. >>>>>> >>>>>> I got a great improvement moving to MQTTSubscribeDriver 2.1.0rc but >>>>>> the problem persists to a small degree. The problem is >>>>>> happening sporadically and rarely. Any thoughts? I really don't want >>>>>> to >>>>>> give up MQTTSubscribeDriver since it is so versatile in what data you >>>>>> can >>>>>> bring into weewx. >>>>>> >>>>>> >>>>>> pi@raspberrypi:~ $ grep "lightning_distance: 7" weewxout1 >>>>>> LOOP: 2021-08-12 13:58:54 EDT (1628791134) active: 1.0, cloudbase: >>>>>> 4673.53199141, dateTime: 1628791134.4, dewpoint: 68.5748592378, >>>>>> heatindex: >>>>>> 84.58478629, humidex: 95.6055479842, lightning_distance: 7.99999999694, >>>>>> lightning_strike_count: None, maxSolarRad: None, outHumidity: 65.0, >>>>>> outTemp: 81.5, rainRate: 0, usUnits: 1 >>>>>> REC: 2021-08-12 14:00:00 EDT (1628791200) active: 0.4, cloudbase: >>>>>> 4592.04983163, dateTime: 1628791200, dewpoint: 68.8533815408, ET: None, >>>>>> heatindex: 84.5975618398, humidex: 95.7627385325, interval: 5, >>>>>> lightning_distance: 7.99999999694, lightning_strike_count: 0.0, >>>>>> maxSolarRad: None, outHumidity: 65.8, outTemp: 81.4200008, rain: 0.04, >>>>>> rainRate: 0.0421052631579, usUnits: 1 >>>>>> LOOP: 2021-08-12 14:42:54 EDT (1628793774) active: 1.0, dateTime: >>>>>> 1628793774.25, lightning_distance: 7.99999999694, maxSolarRad: None, >>>>>> rainRate: 0, usUnits: 1 >>>>>> REC: 2021-08-12 14:45:00 EDT (1628793900) active: 1.0, altimeter: >>>>>> 30.3586782548, barometer: 30.2545352733, cloudbase: 4768.17055948, >>>>>> dateTime: 1628793900, dewpoint: 70.2239033928, ET: None, heatindex: >>>>>> 87.9424503611, humidex: 99.1008122419, interval: 5, lightning_distance: >>>>>> 7.99999999694, lightning_strike_count: 0.0, maxSolarRad: None, >>>>>> outHumidity: >>>>>> 64.3243243243, outTemp: 83.5654538545, pressure: 28.5043137197, rain: >>>>>> 0.0, >>>>>> rainRate: 0.0, usUnits: 1 >>>>>> LOOP: 2021-08-12 14:58:06 EDT (1628794686) active: 1.0, dateTime: >>>>>> 1628794686.48, lightning_distance: 7.99999999694, maxSolarRad: None, >>>>>> rainRate: 0, usUnits: 1 >>>>>> LOOP: 2021-08-12 14:58:30 EDT (1628794710) active: 1.0, dateTime: >>>>>> 1628794710.6, lightning_distance: 7.99999999694, maxSolarRad: None, >>>>>> rainRate: 0, usUnits: 1 >>>>>> REC: 2021-08-12 15:00:00 EDT (1628794800) active: 0.911764705882, >>>>>> altimeter: 30.352863321, barometer: 30.248274916, cloudbase: >>>>>> 4781.8918872, >>>>>> dateTime: 1628794800, dewpoint: 71.158468132, ET: None, heatindex: >>>>>> 89.7957954486, humidex: 100.932534378, interval: 5, lightning_distance: >>>>>> 7.99999999694, lightning_strike_count: 0.0, maxSolarRad: None, >>>>>> outHumidity: >>>>>> 64.3235294118, outTemp: 84.5603924356, pressure: 28.4987883408, rain: >>>>>> 0.0, >>>>>> rainRate: 0.0, usUnits: 1 >>>>>> LOOP: 2021-08-12 15:50:46 EDT (1628797846) active: 1.0, dateTime: >>>>>> 1628797846.38, lightning_distance: 7.99999999694, maxSolarRad: None, >>>>>> rainRate: 0, usUnits: 1 >>>>>> REC: 2021-08-12 15:55:00 EDT (1628798100) active: 1.0, altimeter: >>>>>> 30.3401020979, barometer: 30.2347797117, cloudbase: 4909.75354319, >>>>>> dateTime: 1628798100, dewpoint: 71.8556696944, ET: None, heatindex: >>>>>> 91.9474302989, humidex: 102.838512165, interval: 5, lightning_distance: >>>>>> 7.99999999694, lightning_strike_count: 0.0, maxSolarRad: None, >>>>>> outHumidity: >>>>>> 63.2432432432, outTemp: 85.8181836364, pressure: 28.4866625997, rain: >>>>>> 0.0, >>>>>> rainRate: 0.0, usUnits: 1 >>>>>> >>>>>> On Wednesday, August 11, 2021 at 3:12:22 PM UTC-4 Gene LeSage wrote: >>>>>> >>>>>>> Bingo, fixed by installing v2.1.0-RC MQTTSubscribeDriver. Adding >>>>>>> collect_observations = True and single_queue = True to [[topics], I >>>>>>> get >>>>>>> desired behavior. Null lightning distance is recorded in absence of >>>>>>> strikes and with an test simulated lightning strike by battery charger >>>>>>> shorting, I get lightning strike recording and a lightning distance. >>>>>>> Loop >>>>>>> shows both lightning distance and strikes in same loop: >>>>>>> >>>>>>> LOOP: 2021-08-11 14:56:07 EDT (1628708167) active: 1.0, cloudbase: >>>>>>> 5636.94172822, dateTime: 1628708167.42, dewpoint: 70.6358563958, >>>>>>> heatindex: >>>>>>> 93.4196227576, humidex: 103.698544022, lightning_distance: None, >>>>>>> lightning_strike_count: 0, maxSolarRad: None, outHumidity: 57.0, >>>>>>> outTemp: >>>>>>> 87.8, rainRate: 0, usUnits: 1 >>>>>>> LOOP: 2021-08-11 14:56:07 EDT (1628708167) active: 1.0, cloudbase: >>>>>>> 5636.94172822, dateTime: 1628708167.43, dewpoint: 70.6358563958, >>>>>>> heatindex: >>>>>>> 93.4196227576, humidex: 103.698544022, lightning_distance: None, >>>>>>> lightning_strike_count: 0, maxSolarRad: None, outHumidity: 57.0, >>>>>>> outTemp: >>>>>>> 87.8, rainRate: 0, usUnits: 1 >>>>>>> >>>>>>> Supporting the idea that lightning strikes and distances were being >>>>>>> interpreted at different times I caught two lightning strikes this >>>>>>> morning >>>>>>> (before installing v2.1.0-RC) which shows the 8 am lightning strike >>>>>>> before >>>>>>> lightning distance change and the 11 am lightning strike after >>>>>>> lightning >>>>>>> distance change. By not being synchronized, the correction >>>>>>> (lightning_distance = lightning_distance if lightning_strike_count > 0 >>>>>>> else >>>>>>> None) would never work. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> [image: Screenshot (14).png] >>>>>>> >>>>>>> On Wednesday, August 11, 2021 at 11:41:31 AM UTC-4 [email protected] >>>>>>> wrote: >>>>>>> >>>>>>>> I fear the root cause is that MQTTSubscribeDriver takes each MQTT >>>>>>>> message and creates a loop packet. This works ok for 'aggregated' >>>>>>>> messages >>>>>>>> like json or keyword (the original implementations). But for >>>>>>>> 'individual' >>>>>>>> payloads that are dependent on each other, it is problematic.I've >>>>>>>> already >>>>>>>> had to deal with this for wind data. MQTTSubscribe collects speed and >>>>>>>> direction data into a single loop packet. >>>>>>>> >>>>>>>> Since then I've been experimenting with collecting data and >>>>>>>> creating the loop packet from the collection. Looking at your loop >>>>>>>> packets, I think it might help you. >>>>>>>> >>>>>>>> The code is a pre-release which can be found here, >>>>>>>> https://github.com/bellrichm/WeeWX-MQTTSubscribe/releases/tag/v2.1.0-rc02 >>>>>>>> >>>>>>>> You will need to add two undocumented configuration options. >>>>>>>> >>>>>>>> [MQTTSubscribeDriver] >>>>>>>> . >>>>>>>> . >>>>>>>> . >>>>>>>> >>>>>>>> [[topics] >>>>>>>> # With the exception of wind data, by default a packet is >>>>>>>> created for every MQTT message received. >>>>>>>> # When this is true, MQTTSubscribe attempts to collect >>>>>>>> observations across messages into a packet. >>>>>>>> # Default is False. >>>>>>>> # This is experimental and may be removed. >>>>>>>> collect_observations = True >>>>>>>> >>>>>>>> # With the exception of wind data, by default a queue is >>>>>>>> created for every MQTT topic. >>>>>>>> # When this is true, MQTTSubsribe uses a single queue for >>>>>>>> all non wind data. >>>>>>>> # This is useful when 'collect_observations = True'. >>>>>>>> # Default is False. >>>>>>>> # This is experimental and may be removed. >>>>>>>> single_queue = True >>>>>>>> >>>>>>>> . >>>>>>>> . >>>>>>>> . >>>>>>>> >>>>>>>> With this, the incoming data should be collected and a loop packet >>>>>>>> only created when the arriving observation is already in the >>>>>>>> collection. >>>>>>>> Since, hopefully, the loop packet now has both strike and distance >>>>>>>> data, >>>>>>>> your [[Corrections]] should work. >>>>>>>> >>>>>>>> If this doesn't work. I'll have to go back to the drawing board and >>>>>>>> look for a different solution. If it comes to this, I'd need a debug >>>>>>>> level >>>>>>>> log for at least one archive period so that I can analyze the data. >>>>>>>> >>>>>>>> This is beta code, so the usual caveats apply. >>>>>>>> rich >>>>>>>> >>>>>>>> ps. >>>>>>>> If you are using rtl_433 to publish to MQTT, another option might >>>>>>>> be to use the 'events' topic that it publishes to. This is a json >>>>>>>> formatted >>>>>>>> message so the loop packet created for each MQTT message should have >>>>>>>> both >>>>>>>> the strike and distance data. >>>>>>>> >>>>>>>> On Wednesday, 11 August 2021 at 09:52:39 UTC-4 [email protected] >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I am using weewx-MQTTSubscribe to bring data into Weewx from >>>>>>>>> sensors, so i don't use SDR sensor map, see config for mqttsubscribe >>>>>>>>> here >>>>>>>>> https://github.com/bellrichm/WeeWX-MQTTSubscribe/wiki/Configuring >>>>>>>>> A number of unsuccessful tries to modify mqttsubscribe are commented >>>>>>>>> out. >>>>>>>>> [MQTTSubscribeDriver] >>>>>>>>> [[topics]] >>>>>>>>> >>>>>>>>> [[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/strike_count]]] >>>>>>>>> >>>>>>>>> [[[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/strike_count]]]] >>>>>>>>> name = lightning_strike_count #strikes_total >>>>>>>>> #strike_count >>>>>>>>> contains_total = true >>>>>>>>> # conversion_type = int >>>>>>>>> # units = count >>>>>>>>> # expires_after = 0 >>>>>>>>> >>>>>>>>> [[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/storm_dist]]] >>>>>>>>> >>>>>>>>> [[[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/storm_dist]]]] >>>>>>>>> name = lightning_distance #strike_dist >>>>>>>>> #LS_1_distance >>>>>>>>> # units = mile >>>>>>>>> # conversion_type = int >>>>>>>>> # expires_after = 0 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> [[Corrections]] >>>>>>>>> # For each type, an arbitrary calibration expression can >>>>>>>>> be given. >>>>>>>>> # It should be in the units defined in the StdConvert >>>>>>>>> section. >>>>>>>>> # Example: >>>>>>>>> lightning_distance = lightning_distance if >>>>>>>>> lightning_strike_count > 0 else None >>>>>>>>> >>>>>>>>> [Accumulator] >>>>>>>>> [[lightning_strike_count]] >>>>>>>>> extractor = sum >>>>>>>>> [[lightning_distance]] >>>>>>>>> extractor = min >>>>>>>>> # merger = minmax >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wednesday, August 11, 2021 at 7:58:38 AM UTC-4 >>>>>>>>> [email protected] wrote: >>>>>>>>> >>>>>>>>>> I think going back to your original issue something seems to be >>>>>>>>>> off. Can you post you weewx.conf for lightning data? If you're using >>>>>>>>>> sdr, >>>>>>>>>> should contain the following in each section: >>>>>>>>>> >>>>>>>>>> [SDR] >>>>>>>>>> [[sensor_map]] >>>>>>>>>> ...... >>>>>>>>>> lightning_distance = distance.XXXX.AcuriteLightningPacket >>>>>>>>>> strikes_total = strikes_total.XXXX.AcuriteLightningPacket >>>>>>>>>> [[deltas]] >>>>>>>>>> rain = rain_total >>>>>>>>>> lightning_strike_count = strikes_total >>>>>>>>>> >>>>>>>>>> [StdCalibrate] >>>>>>>>>> >>>>>>>>>> [[Corrections]] >>>>>>>>>> lightning_distance = lightning_distance if >>>>>>>>>> lightning_strike_count > 0 else None >>>>>>>>>> >>>>>>>>>> [Accumulator] >>>>>>>>>> [[lightning_strike_count]] >>>>>>>>>> extractor = sum >>>>>>>>>> [[lightning_distance]] >>>>>>>>>> extractor = min >>>>>>>>>> >>>>>>>>>> On Tuesday, August 10, 2021 at 2:46:44 PM UTC-4 [email protected] >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Initially I thought Rich's solution was going to work, adding >>>>>>>>>>> his suggested correction <lightning_distance = lightning_distance >>>>>>>>>>> if >>>>>>>>>>> 'lightning_strike_count'in locals() and lightning_strike_count> 0 >>>>>>>>>>> else >>>>>>>>>>> None> eliminated my fixed lightning_distance data in the absence >>>>>>>>>>> of >>>>>>>>>>> lightning_counts, but it created a new problem, now I don't get >>>>>>>>>>> lightning_distance values with a lightning strike. I triggered the >>>>>>>>>>> lightning detector with a spark by transiently shorting a battery >>>>>>>>>>> charger >>>>>>>>>>> at time 14:22:16 , which produced 2 detected lightnings and >>>>>>>>>>> 14:22:24 which >>>>>>>>>>> produced one, but the lightning_distance remains at None. My mqtt >>>>>>>>>>> explorer >>>>>>>>>>> reports the lightning detector is sending a constant 12-mile >>>>>>>>>>> lightning >>>>>>>>>>> distant >>>>>>>>>>> >>>>>>>>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: >>>>>>>>>>> 1628619736.82, lightning_distance: None, maxSolarRad: None, >>>>>>>>>>> outHumidity: >>>>>>>>>>> 73.0, rainRate: 0.32, usUnits: 1 >>>>>>>>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: >>>>>>>>>>> 1628619736.81, lightning_strike_count: 2.0, maxSolarRad: None, >>>>>>>>>>> rainRate: >>>>>>>>>>> 0.32, usUnits: 1 >>>>>>>>>>> <-------------------------------------------------------- >>>>>>>>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: >>>>>>>>>>> 1628619736.82, lightning_distance: None, lightning_strike_count: >>>>>>>>>>> 0.0, >>>>>>>>>>> maxSolarRad: None, rainRate: 0.32, usUnits: 1 >>>>>>>>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: >>>>>>>>>>> 1628619736.83, lightning_distance: None, lightning_strike_count: >>>>>>>>>>> 0.0, >>>>>>>>>>> maxSolarRad: None, rainRate: 0.32, usUnits: 1 >>>>>>>>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: >>>>>>>>>>> 1628619736.81, lightning_distance: None, maxSolarRad: None, >>>>>>>>>>> rainRate: 0.32, >>>>>>>>>>> usUnits: 1 >>>>>>>>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: >>>>>>>>>>> 1628619736.82, lightning_distance: None, maxSolarRad: None, >>>>>>>>>>> rainRate: 0.32, >>>>>>>>>>> usUnits: 1 >>>>>>>>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: >>>>>>>>>>> 1628619736.83, lightning_distance: None, maxSolarRad: None, >>>>>>>>>>> rainRate: 0.32, >>>>>>>>>>> usUnits: 1 >>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: >>>>>>>>>>> 1628619744.67, lightning_distance: None, maxSolarRad: None, >>>>>>>>>>> outTemp: >>>>>>>>>>> 82.000004, rainRate: 0.32, usUnits: 1 >>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: >>>>>>>>>>> 1628619744.68, lightning_distance: None, maxSolarRad: None, >>>>>>>>>>> outTemp: >>>>>>>>>>> 82.000004, rainRate: 0.32, usUnits: 1 >>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: >>>>>>>>>>> 1628619744.69, lightning_distance: None, maxSolarRad: None, >>>>>>>>>>> outTemp: >>>>>>>>>>> 82.000004, rainRate: 0.32, usUnits: 1 >>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: >>>>>>>>>>> 1628619744.67, lightning_distance: None, maxSolarRad: None, >>>>>>>>>>> outHumidity: >>>>>>>>>>> 73.0, rainRate: 0.32, usUnits: 1 >>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: >>>>>>>>>>> 1628619744.68, lightning_distance: None, maxSolarRad: None, >>>>>>>>>>> outHumidity: >>>>>>>>>>> 73.0, rainRate: 0.32, usUnits: 1 >>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: >>>>>>>>>>> 1628619744.69, lightning_distance: None, maxSolarRad: None, >>>>>>>>>>> outHumidity: >>>>>>>>>>> 73.0, rainRate: 0.32, usUnits: 1 >>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: >>>>>>>>>>> 1628619744.68, lightning_strike_count: 1.0, maxSolarRad: None, >>>>>>>>>>> rainRate: >>>>>>>>>>> 0.32, usUnits: 1 >>>>>>>>>>> <----------------------------------------------------------- >>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: >>>>>>>>>>> 1628619744.69, lightning_distance: None, lightning_strike_count: >>>>>>>>>>> 0.0, >>>>>>>>>>> maxSolarRad: None, rainRate: 0.32, usUnits: 1 >>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: >>>>>>>>>>> 1628619744.69, lightning_distance: None, lightning_strike_count: >>>>>>>>>>> 0.0, >>>>>>>>>>> maxSolarRad: None, rainRate: 0.32, usUnits: 1 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tuesday, August 10, 2021 at 9:20:17 AM UTC-4 >>>>>>>>>>> [email protected] wrote: >>>>>>>>>>> >>>>>>>>>>>> I'm no python expert (probably know just enough to be >>>>>>>>>>>> dangerous), but something like this might get you want you want. >>>>>>>>>>>> >>>>>>>>>>>> # ligtning_strike_count must exist and have a count > 0 for >>>>>>>>>>>> lightning_distance to have a valid value >>>>>>>>>>>> lightning_distance = lightning_distance if >>>>>>>>>>>> 'lightning_strike_count'in locals() and lightning_strike_count> 0 >>>>>>>>>>>> else None >>>>>>>>>>>> >>>>>>>>>>>> rich >>>>>>>>>>>> On Tuesday, 10 August 2021 at 03:03:01 UTC-4 gjr80 wrote: >>>>>>>>>>>> >>>>>>>>>>>>> On Tuesday, 10 August 2021 at 08:37:42 UTC+10 [email protected] >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Here is my output running weewxd directly, I see three lines >>>>>>>>>>>>>> which show lightning_distance to be None, which is expected from >>>>>>>>>>>>>> corrections, but the following three lines show >>>>>>>>>>>>>> lightning_distance to be >>>>>>>>>>>>>> 11.999... It appears the correction " lightning_distance = >>>>>>>>>>>>>> lightning_distance if lightning_strike_count > 0 else None" is >>>>>>>>>>>>>> working but >>>>>>>>>>>>>> being ignored in final output to graph and database. Any >>>>>>>>>>>>>> additional >>>>>>>>>>>>>> thoughts? >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> The reason you see no change to lightning_distance for some >>>>>>>>>>>>> packets is that for those packets lightning_strike_count does >>>>>>>>>>>>> not exist so your calibration expression fails and no change is >>>>>>>>>>>>> made, in >>>>>>>>>>>>> other words lightning_distance remains as it was >>>>>>>>>>>>> 11.9999999954. Why does this happen, in layman's terms when >>>>>>>>>>>>> lightning_strike_count is not in the loop packet the >>>>>>>>>>>>> calibration expression in effect has an unknown variable ( >>>>>>>>>>>>> lightning_strike_count) and the calibration expression raises >>>>>>>>>>>>> an error. The StdCalibrate service which handles the >>>>>>>>>>>>> calibration expressions catches that error and discards that >>>>>>>>>>>>> calibration >>>>>>>>>>>>> expression. Given the limitations of the StdCalibrate service >>>>>>>>>>>>> I am not aware of any calibration expression that would do as you >>>>>>>>>>>>> want, of >>>>>>>>>>>>> course a WeeWX and python wizard might come along an prove me >>>>>>>>>>>>> wrong! >>>>>>>>>>>>> >>>>>>>>>>>>> My thoughts on a solution, unless there is a simple solution >>>>>>>>>>>>> in your WeeWX driver or elsewhere upstream of WeeWX I would be >>>>>>>>>>>>> writing your >>>>>>>>>>>>> own WeeWX service to look at the packet and make the necessary >>>>>>>>>>>>> correction. >>>>>>>>>>>>> This way you can use a bit of python code to do exactly what you >>>>>>>>>>>>> want and >>>>>>>>>>>>> you won't be limited by the single line expressions as used by >>>>>>>>>>>>> StdCalibrate. All up it should take no more than 10 lines of >>>>>>>>>>>>> code. The service could be a data_service or process_service >>>>>>>>>>>>> (refer weewx.conf [Engine] [[Services]]) but would need to >>>>>>>>>>>>> appear before StdArchive is called. The advantage of your own >>>>>>>>>>>>> service is that there is no need to change any one else's code >>>>>>>>>>>>> (eg WeeWX >>>>>>>>>>>>> driver, upstream code) so you will not have your changes lost >>>>>>>>>>>>> during an >>>>>>>>>>>>> upgrade (WeeWX or otherwise). >>>>>>>>>>>>> >>>>>>>>>>>>> Gary >>>>>>>>>>>>> >>>>>>>>>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/deb99d2b-8bf8-4b0e-8c20-9bca037462e1n%40googlegroups.com.
