In the log should be more info with the error message. I think it might be
multiple lines, which is why it did not show up in the grep.
But, I think the problem is that we don’t have the ‘type’ setting in quite
the correct place. Try something like this
[[topics]]
[[[message]]]
type = json
keyword_separator = ":"
keyword_delimiter = ","
# Units for MQTT payloads without unit value.
# Valid values: US, METRIC, METRICWX
# For more information see,
http://weewx.com/docs/customizing.htm#units
# Default is US
# unit_system = METRIC
[[[homeassistant/sensor/barometer]]]
[[[[message]]]]
type = individual
[[[[homeassistant/sensor/barometer]]]]
name = barometer
units = mbar
Sorry for the tenseness of the replies, I’m away from my computer for a few
days.
rich
On Saturday, 14 August 2021 at 11:53:05 UTC-4 [email protected] wrote:
> Rich, I set up for both json and individual mqtt callback types, the
> beginning of my topics section in [MQTTSubscribeDriver] is shown, the
> remainder of the topic section is the same as my previous post. I am unable
> to get the first topic (homeassistant/sensor/barometer) running as
> individual callback to be recorded in weewx. I ran in logging level =
> DEBUG and looking grep barometer mqtt.log show at bottom, I see that the
> barometer topic is set up, read, with a value of 1025, but the last line
> shows that its value is ignored. Any thoughts why not working?
>
> # [[message_callback]]
> # The format of the MQTT payload.
> # Currently support: individual, json, keyword
> # Must be specified.
> [[topics]]
> [[[message]]]
> type = json
> keyword_separator = ":"
> keyword_delimiter = ","
> # Units for MQTT payloads without unit value.
> # Valid values: US, METRIC, METRICWX
> # For more information see,
> http://weewx.com/docs/customizing.htm#units
> # Default is US
> # unit_system = METRIC
> [[[homeassistant/sensor/barometer]]]
> [[[[homeassistant/sensor/barometer]]]]
> type = individual
> name = barometer
> units = mbar
>
> [[[rtl_433/9b13b3f4-rtl433/events]]]
> msg_id_field = id
> [[[[time]]]]
> ignore_msg_id_field = True
> ignore = True
> [[[[model]]]]
> ignore_msg_id_field = True
> ignore = True
>
> .........
>
>
> grep barometer mqtt.log
> 2021-08-14 11:07:51 DEBUG (Driver) MQTTSUBscriber sanitized_service_dict
> is {u'username': u'----', u'log': u'true', u'topics': {u'message':
> {u'type': u'json', u'keyword_separator': u':', u'keyword_delimiter': u','},
> u'homeassistant/sensor/barometer': {u'homeassistant/sensor/barometer':
> {u'type': u'individual', u'name': u'barometer', u'units': u'mbar',
> u'ignore': u'False'}}, u'rtl_433/9b13b3f4-rtl433/events': {u'msg_id_field':
> u'id', u'time': {u'ignore_msg_id_field': u'True', u'ignore': u'True'},
> u'model': {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'id':
> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'channel':
> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'battery_ok':
> {u'ignore_msg_id_field': u'True', u'ignore': u'True'},
> u'temperature_C_223': {u'name': u'outTemp', u'units': u'degree_C'},
> u'humidity_223': {u'name': u'outHumidity'}, u'strike_count_223': {u'name':
> u'lightning_strike_count', u'contains_total': u'True'}, u'storm_dist_223':
> {u'name': u'lightning_distance', u'units': u'mile'}, u'active':
> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'rfi':
> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'exception':
> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'raw_msg':
> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'mic':
> {u'ignore_msg_id_field': u'True', u'ignore': u'True'}, u'rain_mm_1322':
> {u'name': u'rain', u'contains_total': u'true', u'units': u'mm'}}},
> u'driver': u'user.MQTTSubscribe', u'logging_level': u'DEBUG', u'host':
> u'localhost', u'keepalive': u'60', u'port': u'1883', u'logging_filename':
> u'/home/pi/mqtt.log'}
> 2021-08-14 11:07:51 DEBUG (Driver) TopicManager self.subscribed_topics is
> {"homeassistant/sensor/barometer": {"datetime_format": null,
> "msg_id_field": null, "qos": 0, "filters": {}, "fields":
> {"homeassistant/sensor/barometer": {"name": "barometer", "contains_total":
> false, "conversion_error_to_none": false, "ignore": false,
> "conversion_func": {"source": "lambda x: to_float(x)", "compiled":
> "<function <lambda> at 0x761368b0>"}, "units": "mbar"}},
> "message-1628953671.535918": {"type": "json", "keyword_separator": ":",
> "keyword_delimiter": ","}, "use_server_datetime": false, "queue": {"name":
> "homeassistant/sensor/barometer", "type": "normal", "adjust_start_time":
> 0.0, "adjust_end_time": 0.0, "ignore_start_time": false, "ignore_end_time":
> false, "data": "deque([])", "max_size": 2147483647}, "ignore_msg_id_field":
> [], "ignore": false, "subscribe": true, "conversion_func": {"source":
> "lambda x: to_float(x)", "compiled": "<function <lambda> at 0x76200970>"},
> "unit_system": 1, "topic_tail_is_fieldname": false, "offset_format": null},
> "rtl_433/9b13b3f4-rtl433/events": {"datetime_format": null, "msg_id_field":
> "id", "qos": 0, "filters": {}, "fields": {"exception": {"ignore": true,
> "conversion_func": {"source": "lambda x: to_float(x)", "compiled":
> "<function <lambda> at 0x7614ecb0>"}, "contains_total": false, "name":
> "exception", "conversion_error_to_none": false}, "temperature_C_223":
> {"name": "outTemp", "contains_total": false, "conversion_error_to_none":
> false, "ignore": false, "conversion_func": {"source": "lambda x:
> to_float(x)", "compiled": "<function <lambda> at 0x761366b0>"}, "units":
> "degree_C"}, "rain_mm_1322": {"name": "rain", "contains_total": true,
> "conversion_error_to_none": false, "ignore": false, "conversion_func":
> {"source": "lambda x: to_float(x)", "compiled": "<function <lambda> at
> 0x761530f0>"}, "units": "mm"}, "strike_count_223": {"ignore": false,
> "conversion_func": {"source": "lambda x: to_float(x)", "compiled":
> "<function <lambda> at 0x7614ad70>"}, "contains_total": true, "name":
> "lightning_strike_count", "conversion_error_to_none": false}, "battery_ok":
> {"ignore": true, "conversion_func": {"source": "lambda x: to_float(x)",
> "compiled": "<function <lambda> at 0x76136bb0>"}, "contains_total": false,
> "name": "battery_ok", "conversion_error_to_none": false}, "humidity_223":
> {"ignore": false, "conversion_func": {"source": "lambda x: to_float(x)",
> "compiled": "<function <lambda> at 0x7614a9b0>"}, "contains_total": false,
> "name": "outHumidity", "conversion_error_to_none": false}, "mic":
> {"ignore": true, "conversion_func": {"source": "lambda x: to_float(x)",
> "compiled": "<function <lambda> at 0x7614e3f0>"}, "contains_total": false,
> "name": "mic", "conversion_error_to_none": false}, "raw_msg": {"ignore":
> true, "conversion_func": {"source": "lambda x: to_float(x)", "compiled":
> "<function <lambda> at 0x7614ebb0>"}, "contains_total": false, "name":
> "raw_msg", "conversion_error_to_none": false}, "rfi": {"ignore": true,
> "conversion_func": {"source": "lambda x: to_float(x)", "compiled":
> "<function <lambda> at 0x7614eb70>"}, "contains_total": false, "name":
> "rfi", "conversion_error_to_none": false}, "storm_dist_223": {"name":
> "lightning_distance", "contains_total": false, "conversion_error_to_none":
> false, "ignore": false, "conversion_func": {"source": "lambda x:
> to_float(x)", "compiled": "<function <lambda> at 0x7614a770>"}, "units":
> "mile"}, "time": {"ignore": true, "conversion_func": {"source": "lambda x:
> to_float(x)", "compiled": "<function <lambda> at 0x761367f0>"},
> "contains_total": false, "name": "time", "conversion_error_to_none":
> false}, "active": {"ignore": true, "conversion_func": {"source": "lambda x:
> to_float(x)", "compiled": "<function <lambda> at 0x7614e730>"},
> "contains_total": false, "name": "active", "conversion_error_to_none":
> false}, "model": {"ignore": true, "conversion_func": {"source": "lambda x:
> to_float(x)", "compiled": "<function <lambda> at 0x761367b0>"},
> "contains_total": false, "name": "model", "conversion_error_to_none":
> false}, "id": {"ignore": true, "conversion_func": {"source": "lambda x:
> to_float(x)", "compiled": "<function <lambda> at 0x76136730>"},
> "contains_total": false, "name": "id", "conversion_error_to_none": false},
> "channel": {"ignore": true, "conversion_func": {"source": "lambda x:
> to_float(x)", "compiled": "<function <lambda> at 0x76136370>"},
> "contains_total": false, "name": "channel", "conversion_error_to_none":
> false}}, "message-1628953671.535918": {"type": "json", "keyword_separator":
> ":", "keyword_delimiter": ","}, "use_server_datetime": false, "queue":
> {"name": "rtl_433/9b13b3f4-rtl433/events", "type": "normal",
> "adjust_start_time": 0.0, "adjust_end_time": 0.0, "ignore_start_time":
> false, "ignore_end_time": false, "data": "deque([])", "max_size":
> 2147483647}, "ignore_msg_id_field": ["time", "model", "id", "channel",
> "battery_ok", "active", "rfi", "exception", "raw_msg", "mic"], "ignore":
> false, "subscribe": true, "conversion_func": {"source": "lambda x:
> to_float(x)", "compiled": "<function <lambda> at 0x761365b0>"},
> "unit_system": 1, "topic_tail_is_fieldname": false, "offset_format": null},
> "1628953671.557307-windGust-windGustDir-windDir-windSpeed":
> {"datetime_format": null, "queue": {"name":
> "1628953671.557307-windGust-windGustDir-windDir-windSpeed", "type":
> "collector", "adjust_start_time": 0.0, "adjust_end_time": 0.0,
> "ignore_start_time": false, "ignore_end_time": false, "data": "deque([])",
> "max_size": 2147483647}, "qos": 0, "message-1628953671.535918": {},
> "use_server_datetime": false, "offset_format": null, "subscribe": false,
> "unit_system": 1, "topic_tail_is_fieldname": false}}
> 2021-08-14 11:07:51 DEBUG (Driver) MQTTSubscribe MQTT: Sending SUBSCRIBE
> (d0, m1) [('homeassistant/sensor/barometer', 0)]
> 2021-08-14 11:07:51 INFO (Driver) Subscribing to
> homeassistant/sensor/barometer has a mid 1 and rc 0
> 2021-08-14 11:10:02 DEBUG (Driver) MQTTSubscribe MQTT: Received PUBLISH
> (d0, q0, r0, m0), 'homeassistant/sensor/barometer', ... (8 bytes)
> 2021-08-14 11:10:02 DEBUG (Driver) MessageCallbackProvider data-> incoming
> topic: homeassistant/sensor/barometer, QOS: 0, retain: 0, payload: 1025.455
> 2021-08-14 11:10:02 ERROR (Driver) **** MessageCallbackProvider Ignoring
> topic=homeassistant/sensor/barometer and payload=1025.455
>
> On Friday, August 13, 2021 at 9:53:04 PM UTC-4 [email protected] wrote:
>
>> Good to hear you have something working.
>> Here is some information on configuring the message type by payload.
>>
>> https://github.com/bellrichm/WeeWX-MQTTSubscribe/issues/113#issuecomment-809655246
>> This exists on the version you have. If you have any feedback, let me
>> know.
>> rich
>>
>>
>> On Friday, 13 August 2021 at 17:03:28 UTC-4 [email protected] wrote:
>>
>>> Rich, I have solved problem for persisting distance values without
>>> lightning strikes, I am now using json rather than individual message
>>> callback, this leads to both lightning_strikes and lightning_distance to be
>>> in same loop which as you predicted would solved the problem. My
>>> [MQTTSubscribeDriver] entry for the acurite 6054 lightning detector and
>>> acurite -rain899 rainfall gauge are included below, my example shows
>>> working for msg_id_field for acurite 6054 to be 233 and the acurite
>>> -rain899 to be 1322. Msg_id_field may differ for other peoples devices,
>>> but I don't know.
>>>
>>> One question remains, as usual when you solve one problem, you create
>>> another, I have sensors which for their mqtt output are not formatted in
>>> json, but work using "individual" callback. Some might be convertible to
>>> json but others will not. I see you have in configuring experimental
>>> options, the option to pick for each topic the type of format, with "name =
>>> value format" but I am not sure where that goes or if it will work with my
>>> current version (2.1.0rc). Thanks for the suggestions and the great work
>>> on MQTTSubscribeDriver. Gene
>>>
>>>
>>>
>>>
>>> [[message_callback]]
>>> # The format of the MQTT payload.
>>> # Currently support: individual, json, keyword
>>> # Must be specified.
>>> type = json
>>> # keyword_separator = ":"
>>> # keyword_delimiter = ","
>>> [[topics]]
>>> # Units for MQTT payloads without unit value.
>>> # Valid values: US, METRIC, METRICWX
>>> # For more information see,
>>> http://weewx.com/docs/customizing.htm#units
>>> # Default is US
>>> unit_system = METRIC
>>> [[[rtl_433/9b13b3f4-rtl433/events]]]
>>> msg_id_field = id
>>> [[[[time]]]]
>>> ignore_msg_id_field = True
>>> ignore = True
>>> [[[[model]]]]
>>> ignore_msg_id_field = True
>>> ignore = True
>>> [[[[id]]]]
>>> ignore_msg_id_field = True
>>> ignore = True
>>> [[[[channel]]]]
>>> ignore_msg_id_field = True
>>> ignore = True
>>> [[[[battery_ok]]]]
>>> ignore_msg_id_field = True
>>> ignore = True
>>> [[[[temperature_C_223]]]]
>>> name = outTemp
>>> units = degree_C
>>> [[[[humidity_223]]]]
>>> name = outHumidity
>>> [[[[strike_count_223]]]]
>>> name = lightning_strike_count
>>> contains_total = True
>>> [[[[storm_dist_223]]]]
>>> name = lightning_distance
>>> units = mile
>>> [[[[active]]]]
>>> ignore_msg_id_field = True
>>> ignore = True
>>> [[[[rfi]]]]
>>> ignore_msg_id_field = True
>>> ignore = True
>>> [[[[exception]]]]
>>> ignore_msg_id_field = True
>>> ignore = True
>>> [[[[raw_msg]]]]
>>> ignore_msg_id_field = True
>>> ignore = True
>>> [[[[mic]]]]
>>> ignore_msg_id_field = True
>>> ignore = True
>>> [[[[rain_mm_1322]]]]
>>> name = rain
>>> contains_total = true
>>> units = mm
>>>
>>>
>>> On Thursday, August 12, 2021 at 7:49:04 PM UTC-4 [email protected]
>>> wrote:
>>>
>>>> I am guessing that you are getting loop packets with just distance
>>>> because of the order the data is arriving over MQTT.
>>>>
>>>> For example:
>>>> message with distance arrives
>>>> - data is saved
>>>> another message with distance arrives
>>>> - packet with saved data is created
>>>> - current message is saved
>>>> message with count arrives
>>>> - count is saved
>>>> a message with either distance or count arrives
>>>> - packet with saved data is created
>>>> - data received is saved
>>>>
>>>> A debug level log would confirm this.
>>>>
>>>> If a ‘corrections’ expression won’t fix this, I will need a debug level
>>>> log to get an understanding of the data being received.
>>>>
>>>> If you are using rtl_433 to publish to MQTT, I’d also be interested in
>>>> what is being published to the ‘events’ topic.
>>>> rich
>>>>
>>>> On Thursday, 12 August 2021 at 16:45:35 UTC-4 [email protected] wrote:
>>>>
>>>>> Not quite bingo, I am still getting some fixed lightning_distance
>>>>> integers in the absence of strikes, about 6 per hour. Loops are produced
>>>>> with lightning_distance = 7.999 and no information on
>>>>> lightning_strike_count and other loops have with both lightning_distance
>>>>> =
>>>>> 7.999 and lightning_strike_count = 0 See below output from weewxd for 2
>>>>> hours.
>>>>>
>>>>> I changed [[Corrections]] to
>>>>> lightning_distance = None if lightning_strike_count == 0 or None
>>>>> else lightning_distance
>>>>>
>>>>> thinking that this correction would set lightning_distance to none in
>>>>> the absence of a lightning strike count, but the correction had no
>>>>> effect.
>>>>> Looking at all the loops there is no reason some loops have
>>>>> lightning_distance: 7.99 while the great majority have the desired
>>>>> lightning_distance: None.
>>>>>
>>>>> I got a great improvement moving to MQTTSubscribeDriver 2.1.0rc but
>>>>> the problem persists to a small degree. The problem is
>>>>> happening sporadically and rarely. Any thoughts? I really don't want
>>>>> to
>>>>> give up MQTTSubscribeDriver since it is so versatile in what data you can
>>>>> bring into weewx.
>>>>>
>>>>>
>>>>> pi@raspberrypi:~ $ grep "lightning_distance: 7" weewxout1
>>>>> LOOP: 2021-08-12 13:58:54 EDT (1628791134) active: 1.0, cloudbase:
>>>>> 4673.53199141, dateTime: 1628791134.4, dewpoint: 68.5748592378,
>>>>> heatindex:
>>>>> 84.58478629, humidex: 95.6055479842, lightning_distance: 7.99999999694,
>>>>> lightning_strike_count: None, maxSolarRad: None, outHumidity: 65.0,
>>>>> outTemp: 81.5, rainRate: 0, usUnits: 1
>>>>> REC: 2021-08-12 14:00:00 EDT (1628791200) active: 0.4, cloudbase:
>>>>> 4592.04983163, dateTime: 1628791200, dewpoint: 68.8533815408, ET: None,
>>>>> heatindex: 84.5975618398, humidex: 95.7627385325, interval: 5,
>>>>> lightning_distance: 7.99999999694, lightning_strike_count: 0.0,
>>>>> maxSolarRad: None, outHumidity: 65.8, outTemp: 81.4200008, rain: 0.04,
>>>>> rainRate: 0.0421052631579, usUnits: 1
>>>>> LOOP: 2021-08-12 14:42:54 EDT (1628793774) active: 1.0, dateTime:
>>>>> 1628793774.25, lightning_distance: 7.99999999694, maxSolarRad: None,
>>>>> rainRate: 0, usUnits: 1
>>>>> REC: 2021-08-12 14:45:00 EDT (1628793900) active: 1.0, altimeter:
>>>>> 30.3586782548, barometer: 30.2545352733, cloudbase: 4768.17055948,
>>>>> dateTime: 1628793900, dewpoint: 70.2239033928, ET: None, heatindex:
>>>>> 87.9424503611, humidex: 99.1008122419, interval: 5, lightning_distance:
>>>>> 7.99999999694, lightning_strike_count: 0.0, maxSolarRad: None,
>>>>> outHumidity:
>>>>> 64.3243243243, outTemp: 83.5654538545, pressure: 28.5043137197, rain:
>>>>> 0.0,
>>>>> rainRate: 0.0, usUnits: 1
>>>>> LOOP: 2021-08-12 14:58:06 EDT (1628794686) active: 1.0, dateTime:
>>>>> 1628794686.48, lightning_distance: 7.99999999694, maxSolarRad: None,
>>>>> rainRate: 0, usUnits: 1
>>>>> LOOP: 2021-08-12 14:58:30 EDT (1628794710) active: 1.0, dateTime:
>>>>> 1628794710.6, lightning_distance: 7.99999999694, maxSolarRad: None,
>>>>> rainRate: 0, usUnits: 1
>>>>> REC: 2021-08-12 15:00:00 EDT (1628794800) active: 0.911764705882,
>>>>> altimeter: 30.352863321, barometer: 30.248274916, cloudbase:
>>>>> 4781.8918872,
>>>>> dateTime: 1628794800, dewpoint: 71.158468132, ET: None, heatindex:
>>>>> 89.7957954486, humidex: 100.932534378, interval: 5, lightning_distance:
>>>>> 7.99999999694, lightning_strike_count: 0.0, maxSolarRad: None,
>>>>> outHumidity:
>>>>> 64.3235294118, outTemp: 84.5603924356, pressure: 28.4987883408, rain:
>>>>> 0.0,
>>>>> rainRate: 0.0, usUnits: 1
>>>>> LOOP: 2021-08-12 15:50:46 EDT (1628797846) active: 1.0, dateTime:
>>>>> 1628797846.38, lightning_distance: 7.99999999694, maxSolarRad: None,
>>>>> rainRate: 0, usUnits: 1
>>>>> REC: 2021-08-12 15:55:00 EDT (1628798100) active: 1.0, altimeter:
>>>>> 30.3401020979, barometer: 30.2347797117, cloudbase: 4909.75354319,
>>>>> dateTime: 1628798100, dewpoint: 71.8556696944, ET: None, heatindex:
>>>>> 91.9474302989, humidex: 102.838512165, interval: 5, lightning_distance:
>>>>> 7.99999999694, lightning_strike_count: 0.0, maxSolarRad: None,
>>>>> outHumidity:
>>>>> 63.2432432432, outTemp: 85.8181836364, pressure: 28.4866625997, rain:
>>>>> 0.0,
>>>>> rainRate: 0.0, usUnits: 1
>>>>>
>>>>> On Wednesday, August 11, 2021 at 3:12:22 PM UTC-4 Gene LeSage wrote:
>>>>>
>>>>>> Bingo, fixed by installing v2.1.0-RC MQTTSubscribeDriver. Adding
>>>>>> collect_observations = True and single_queue = True to [[topics], I
>>>>>> get
>>>>>> desired behavior. Null lightning distance is recorded in absence of
>>>>>> strikes and with an test simulated lightning strike by battery charger
>>>>>> shorting, I get lightning strike recording and a lightning distance.
>>>>>> Loop
>>>>>> shows both lightning distance and strikes in same loop:
>>>>>>
>>>>>> LOOP: 2021-08-11 14:56:07 EDT (1628708167) active: 1.0, cloudbase:
>>>>>> 5636.94172822, dateTime: 1628708167.42, dewpoint: 70.6358563958,
>>>>>> heatindex:
>>>>>> 93.4196227576, humidex: 103.698544022, lightning_distance: None,
>>>>>> lightning_strike_count: 0, maxSolarRad: None, outHumidity: 57.0,
>>>>>> outTemp:
>>>>>> 87.8, rainRate: 0, usUnits: 1
>>>>>> LOOP: 2021-08-11 14:56:07 EDT (1628708167) active: 1.0, cloudbase:
>>>>>> 5636.94172822, dateTime: 1628708167.43, dewpoint: 70.6358563958,
>>>>>> heatindex:
>>>>>> 93.4196227576, humidex: 103.698544022, lightning_distance: None,
>>>>>> lightning_strike_count: 0, maxSolarRad: None, outHumidity: 57.0,
>>>>>> outTemp:
>>>>>> 87.8, rainRate: 0, usUnits: 1
>>>>>>
>>>>>> Supporting the idea that lightning strikes and distances were being
>>>>>> interpreted at different times I caught two lightning strikes this
>>>>>> morning
>>>>>> (before installing v2.1.0-RC) which shows the 8 am lightning strike
>>>>>> before
>>>>>> lightning distance change and the 11 am lightning strike after
>>>>>> lightning
>>>>>> distance change. By not being synchronized, the correction
>>>>>> (lightning_distance = lightning_distance if lightning_strike_count > 0
>>>>>> else
>>>>>> None) would never work.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> [image: Screenshot (14).png]
>>>>>>
>>>>>> On Wednesday, August 11, 2021 at 11:41:31 AM UTC-4 [email protected]
>>>>>> wrote:
>>>>>>
>>>>>>> I fear the root cause is that MQTTSubscribeDriver takes each MQTT
>>>>>>> message and creates a loop packet. This works ok for 'aggregated'
>>>>>>> messages
>>>>>>> like json or keyword (the original implementations). But for
>>>>>>> 'individual'
>>>>>>> payloads that are dependent on each other, it is problematic.I've
>>>>>>> already
>>>>>>> had to deal with this for wind data. MQTTSubscribe collects speed and
>>>>>>> direction data into a single loop packet.
>>>>>>>
>>>>>>> Since then I've been experimenting with collecting data and creating
>>>>>>> the loop packet from the collection. Looking at your loop packets, I
>>>>>>> think
>>>>>>> it might help you.
>>>>>>>
>>>>>>> The code is a pre-release which can be found here,
>>>>>>> https://github.com/bellrichm/WeeWX-MQTTSubscribe/releases/tag/v2.1.0-rc02
>>>>>>>
>>>>>>> You will need to add two undocumented configuration options.
>>>>>>>
>>>>>>> [MQTTSubscribeDriver]
>>>>>>> .
>>>>>>> .
>>>>>>> .
>>>>>>>
>>>>>>> [[topics]
>>>>>>> # With the exception of wind data, by default a packet is
>>>>>>> created for every MQTT message received.
>>>>>>> # When this is true, MQTTSubscribe attempts to collect
>>>>>>> observations across messages into a packet.
>>>>>>> # Default is False.
>>>>>>> # This is experimental and may be removed.
>>>>>>> collect_observations = True
>>>>>>>
>>>>>>> # With the exception of wind data, by default a queue is
>>>>>>> created for every MQTT topic.
>>>>>>> # When this is true, MQTTSubsribe uses a single queue for
>>>>>>> all non wind data.
>>>>>>> # This is useful when 'collect_observations = True'.
>>>>>>> # Default is False.
>>>>>>> # This is experimental and may be removed.
>>>>>>> single_queue = True
>>>>>>>
>>>>>>> .
>>>>>>> .
>>>>>>> .
>>>>>>>
>>>>>>> With this, the incoming data should be collected and a loop packet
>>>>>>> only created when the arriving observation is already in the
>>>>>>> collection.
>>>>>>> Since, hopefully, the loop packet now has both strike and distance
>>>>>>> data,
>>>>>>> your [[Corrections]] should work.
>>>>>>>
>>>>>>> If this doesn't work. I'll have to go back to the drawing board and
>>>>>>> look for a different solution. If it comes to this, I'd need a debug
>>>>>>> level
>>>>>>> log for at least one archive period so that I can analyze the data.
>>>>>>>
>>>>>>> This is beta code, so the usual caveats apply.
>>>>>>> rich
>>>>>>>
>>>>>>> ps.
>>>>>>> If you are using rtl_433 to publish to MQTT, another option might be
>>>>>>> to use the 'events' topic that it publishes to. This is a json
>>>>>>> formatted
>>>>>>> message so the loop packet created for each MQTT message should have
>>>>>>> both
>>>>>>> the strike and distance data.
>>>>>>>
>>>>>>> On Wednesday, 11 August 2021 at 09:52:39 UTC-4 [email protected]
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I am using weewx-MQTTSubscribe to bring data into Weewx from
>>>>>>>> sensors, so i don't use SDR sensor map, see config for mqttsubscribe
>>>>>>>> here
>>>>>>>> https://github.com/bellrichm/WeeWX-MQTTSubscribe/wiki/Configuring
>>>>>>>> A number of unsuccessful tries to modify mqttsubscribe are commented
>>>>>>>> out.
>>>>>>>> [MQTTSubscribeDriver]
>>>>>>>> [[topics]]
>>>>>>>>
>>>>>>>> [[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/strike_count]]]
>>>>>>>>
>>>>>>>> [[[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/strike_count]]]]
>>>>>>>> name = lightning_strike_count #strikes_total
>>>>>>>> #strike_count
>>>>>>>> contains_total = true
>>>>>>>> # conversion_type = int
>>>>>>>> # units = count
>>>>>>>> # expires_after = 0
>>>>>>>>
>>>>>>>> [[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/storm_dist]]]
>>>>>>>>
>>>>>>>> [[[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/storm_dist]]]]
>>>>>>>> name = lightning_distance #strike_dist
>>>>>>>> #LS_1_distance
>>>>>>>> # units = mile
>>>>>>>> # conversion_type = int
>>>>>>>> # expires_after = 0
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> [[Corrections]]
>>>>>>>> # For each type, an arbitrary calibration expression can be
>>>>>>>> given.
>>>>>>>> # It should be in the units defined in the StdConvert
>>>>>>>> section.
>>>>>>>> # Example:
>>>>>>>> lightning_distance = lightning_distance if
>>>>>>>> lightning_strike_count > 0 else None
>>>>>>>>
>>>>>>>> [Accumulator]
>>>>>>>> [[lightning_strike_count]]
>>>>>>>> extractor = sum
>>>>>>>> [[lightning_distance]]
>>>>>>>> extractor = min
>>>>>>>> # merger = minmax
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wednesday, August 11, 2021 at 7:58:38 AM UTC-4
>>>>>>>> [email protected] wrote:
>>>>>>>>
>>>>>>>>> I think going back to your original issue something seems to be
>>>>>>>>> off. Can you post you weewx.conf for lightning data? If you're using
>>>>>>>>> sdr,
>>>>>>>>> should contain the following in each section:
>>>>>>>>>
>>>>>>>>> [SDR]
>>>>>>>>> [[sensor_map]]
>>>>>>>>> ......
>>>>>>>>> lightning_distance = distance.XXXX.AcuriteLightningPacket
>>>>>>>>> strikes_total = strikes_total.XXXX.AcuriteLightningPacket
>>>>>>>>> [[deltas]]
>>>>>>>>> rain = rain_total
>>>>>>>>> lightning_strike_count = strikes_total
>>>>>>>>>
>>>>>>>>> [StdCalibrate]
>>>>>>>>>
>>>>>>>>> [[Corrections]]
>>>>>>>>> lightning_distance = lightning_distance if
>>>>>>>>> lightning_strike_count > 0 else None
>>>>>>>>>
>>>>>>>>> [Accumulator]
>>>>>>>>> [[lightning_strike_count]]
>>>>>>>>> extractor = sum
>>>>>>>>> [[lightning_distance]]
>>>>>>>>> extractor = min
>>>>>>>>>
>>>>>>>>> On Tuesday, August 10, 2021 at 2:46:44 PM UTC-4 [email protected]
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Initially I thought Rich's solution was going to work, adding his
>>>>>>>>>> suggested correction <lightning_distance = lightning_distance if
>>>>>>>>>> 'lightning_strike_count'in locals() and lightning_strike_count> 0
>>>>>>>>>> else
>>>>>>>>>> None> eliminated my fixed lightning_distance data in the absence of
>>>>>>>>>> lightning_counts, but it created a new problem, now I don't get
>>>>>>>>>> lightning_distance values with a lightning strike. I triggered the
>>>>>>>>>> lightning detector with a spark by transiently shorting a battery
>>>>>>>>>> charger
>>>>>>>>>> at time 14:22:16 , which produced 2 detected lightnings and 14:22:24
>>>>>>>>>> which
>>>>>>>>>> produced one, but the lightning_distance remains at None. My mqtt
>>>>>>>>>> explorer
>>>>>>>>>> reports the lightning detector is sending a constant 12-mile
>>>>>>>>>> lightning
>>>>>>>>>> distant
>>>>>>>>>>
>>>>>>>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime:
>>>>>>>>>> 1628619736.82, lightning_distance: None, maxSolarRad: None,
>>>>>>>>>> outHumidity:
>>>>>>>>>> 73.0, rainRate: 0.32, usUnits: 1
>>>>>>>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime:
>>>>>>>>>> 1628619736.81, lightning_strike_count: 2.0, maxSolarRad: None,
>>>>>>>>>> rainRate:
>>>>>>>>>> 0.32, usUnits: 1
>>>>>>>>>> <--------------------------------------------------------
>>>>>>>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime:
>>>>>>>>>> 1628619736.82, lightning_distance: None, lightning_strike_count:
>>>>>>>>>> 0.0,
>>>>>>>>>> maxSolarRad: None, rainRate: 0.32, usUnits: 1
>>>>>>>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime:
>>>>>>>>>> 1628619736.83, lightning_distance: None, lightning_strike_count:
>>>>>>>>>> 0.0,
>>>>>>>>>> maxSolarRad: None, rainRate: 0.32, usUnits: 1
>>>>>>>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime:
>>>>>>>>>> 1628619736.81, lightning_distance: None, maxSolarRad: None,
>>>>>>>>>> rainRate: 0.32,
>>>>>>>>>> usUnits: 1
>>>>>>>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime:
>>>>>>>>>> 1628619736.82, lightning_distance: None, maxSolarRad: None,
>>>>>>>>>> rainRate: 0.32,
>>>>>>>>>> usUnits: 1
>>>>>>>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime:
>>>>>>>>>> 1628619736.83, lightning_distance: None, maxSolarRad: None,
>>>>>>>>>> rainRate: 0.32,
>>>>>>>>>> usUnits: 1
>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime:
>>>>>>>>>> 1628619744.67, lightning_distance: None, maxSolarRad: None, outTemp:
>>>>>>>>>> 82.000004, rainRate: 0.32, usUnits: 1
>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime:
>>>>>>>>>> 1628619744.68, lightning_distance: None, maxSolarRad: None, outTemp:
>>>>>>>>>> 82.000004, rainRate: 0.32, usUnits: 1
>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime:
>>>>>>>>>> 1628619744.69, lightning_distance: None, maxSolarRad: None, outTemp:
>>>>>>>>>> 82.000004, rainRate: 0.32, usUnits: 1
>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime:
>>>>>>>>>> 1628619744.67, lightning_distance: None, maxSolarRad: None,
>>>>>>>>>> outHumidity:
>>>>>>>>>> 73.0, rainRate: 0.32, usUnits: 1
>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime:
>>>>>>>>>> 1628619744.68, lightning_distance: None, maxSolarRad: None,
>>>>>>>>>> outHumidity:
>>>>>>>>>> 73.0, rainRate: 0.32, usUnits: 1
>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime:
>>>>>>>>>> 1628619744.69, lightning_distance: None, maxSolarRad: None,
>>>>>>>>>> outHumidity:
>>>>>>>>>> 73.0, rainRate: 0.32, usUnits: 1
>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime:
>>>>>>>>>> 1628619744.68, lightning_strike_count: 1.0, maxSolarRad: None,
>>>>>>>>>> rainRate:
>>>>>>>>>> 0.32, usUnits: 1
>>>>>>>>>> <-----------------------------------------------------------
>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime:
>>>>>>>>>> 1628619744.69, lightning_distance: None, lightning_strike_count:
>>>>>>>>>> 0.0,
>>>>>>>>>> maxSolarRad: None, rainRate: 0.32, usUnits: 1
>>>>>>>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime:
>>>>>>>>>> 1628619744.69, lightning_distance: None, lightning_strike_count:
>>>>>>>>>> 0.0,
>>>>>>>>>> maxSolarRad: None, rainRate: 0.32, usUnits: 1
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tuesday, August 10, 2021 at 9:20:17 AM UTC-4 [email protected]
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I'm no python expert (probably know just enough to be
>>>>>>>>>>> dangerous), but something like this might get you want you want.
>>>>>>>>>>>
>>>>>>>>>>> # ligtning_strike_count must exist and have a count > 0 for
>>>>>>>>>>> lightning_distance to have a valid value
>>>>>>>>>>> lightning_distance = lightning_distance if
>>>>>>>>>>> 'lightning_strike_count'in locals() and lightning_strike_count> 0
>>>>>>>>>>> else None
>>>>>>>>>>>
>>>>>>>>>>> rich
>>>>>>>>>>> On Tuesday, 10 August 2021 at 03:03:01 UTC-4 gjr80 wrote:
>>>>>>>>>>>
>>>>>>>>>>>> On Tuesday, 10 August 2021 at 08:37:42 UTC+10 [email protected]
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Here is my output running weewxd directly, I see three lines
>>>>>>>>>>>>> which show lightning_distance to be None, which is expected from
>>>>>>>>>>>>> corrections, but the following three lines show
>>>>>>>>>>>>> lightning_distance to be
>>>>>>>>>>>>> 11.999... It appears the correction " lightning_distance =
>>>>>>>>>>>>> lightning_distance if lightning_strike_count > 0 else None" is
>>>>>>>>>>>>> working but
>>>>>>>>>>>>> being ignored in final output to graph and database. Any
>>>>>>>>>>>>> additional
>>>>>>>>>>>>> thoughts?
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> The reason you see no change to lightning_distance for some
>>>>>>>>>>>> packets is that for those packets lightning_strike_count does
>>>>>>>>>>>> not exist so your calibration expression fails and no change is
>>>>>>>>>>>> made, in
>>>>>>>>>>>> other words lightning_distance remains as it was
>>>>>>>>>>>> 11.9999999954. Why does this happen, in layman's terms when
>>>>>>>>>>>> lightning_strike_count is not in the loop packet the
>>>>>>>>>>>> calibration expression in effect has an unknown variable (
>>>>>>>>>>>> lightning_strike_count) and the calibration expression raises
>>>>>>>>>>>> an error. The StdCalibrate service which handles the
>>>>>>>>>>>> calibration expressions catches that error and discards that
>>>>>>>>>>>> calibration
>>>>>>>>>>>> expression. Given the limitations of the StdCalibrate service
>>>>>>>>>>>> I am not aware of any calibration expression that would do as you
>>>>>>>>>>>> want, of
>>>>>>>>>>>> course a WeeWX and python wizard might come along an prove me
>>>>>>>>>>>> wrong!
>>>>>>>>>>>>
>>>>>>>>>>>> My thoughts on a solution, unless there is a simple solution in
>>>>>>>>>>>> your WeeWX driver or elsewhere upstream of WeeWX I would be
>>>>>>>>>>>> writing your
>>>>>>>>>>>> own WeeWX service to look at the packet and make the necessary
>>>>>>>>>>>> correction.
>>>>>>>>>>>> This way you can use a bit of python code to do exactly what you
>>>>>>>>>>>> want and
>>>>>>>>>>>> you won't be limited by the single line expressions as used by
>>>>>>>>>>>> StdCalibrate. All up it should take no more than 10 lines of
>>>>>>>>>>>> code. The service could be a data_service or process_service
>>>>>>>>>>>> (refer weewx.conf [Engine] [[Services]]) but would need to
>>>>>>>>>>>> appear before StdArchive is called. The advantage of your own
>>>>>>>>>>>> service is that there is no need to change any one else's code (eg
>>>>>>>>>>>> WeeWX
>>>>>>>>>>>> driver, upstream code) so you will not have your changes lost
>>>>>>>>>>>> during an
>>>>>>>>>>>> upgrade (WeeWX or otherwise).
>>>>>>>>>>>>
>>>>>>>>>>>> Gary
>>>>>>>>>>>>
>>>>>>>>>>>
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/64f26af3-72fa-48cb-86d1-7c258fe9ef3cn%40googlegroups.com.
