Not quite bingo, I am still getting some fixed lightning_distance integers in the absence of strikes, about 6 per hour. Loops are produced with lightning_distance = 7.999 and no information on lightning_strike_count and other loops have with both lightning_distance = 7.999 and lightning_strike_count = 0 See below output from weewxd for 2 hours.
I changed [[Corrections]] to lightning_distance = None if lightning_strike_count == 0 or None else lightning_distance thinking that this correction would set lightning_distance to none in the absence of a lightning strike count, but the correction had no effect. Looking at all the loops there is no reason some loops have lightning_distance: 7.99 while the great majority have the desired lightning_distance: None. I got a great improvement moving to MQTTSubscribeDriver 2.1.0rc but the problem persists to a small degree. The problem is happening sporadically and rarely. Any thoughts? I really don't want to give up MQTTSubscribeDriver since it is so versatile in what data you can bring into weewx. pi@raspberrypi:~ $ grep "lightning_distance: 7" weewxout1 LOOP: 2021-08-12 13:58:54 EDT (1628791134) active: 1.0, cloudbase: 4673.53199141, dateTime: 1628791134.4, dewpoint: 68.5748592378, heatindex: 84.58478629, humidex: 95.6055479842, lightning_distance: 7.99999999694, lightning_strike_count: None, maxSolarRad: None, outHumidity: 65.0, outTemp: 81.5, rainRate: 0, usUnits: 1 REC: 2021-08-12 14:00:00 EDT (1628791200) active: 0.4, cloudbase: 4592.04983163, dateTime: 1628791200, dewpoint: 68.8533815408, ET: None, heatindex: 84.5975618398, humidex: 95.7627385325, interval: 5, lightning_distance: 7.99999999694, lightning_strike_count: 0.0, maxSolarRad: None, outHumidity: 65.8, outTemp: 81.4200008, rain: 0.04, rainRate: 0.0421052631579, usUnits: 1 LOOP: 2021-08-12 14:42:54 EDT (1628793774) active: 1.0, dateTime: 1628793774.25, lightning_distance: 7.99999999694, maxSolarRad: None, rainRate: 0, usUnits: 1 REC: 2021-08-12 14:45:00 EDT (1628793900) active: 1.0, altimeter: 30.3586782548, barometer: 30.2545352733, cloudbase: 4768.17055948, dateTime: 1628793900, dewpoint: 70.2239033928, ET: None, heatindex: 87.9424503611, humidex: 99.1008122419, interval: 5, lightning_distance: 7.99999999694, lightning_strike_count: 0.0, maxSolarRad: None, outHumidity: 64.3243243243, outTemp: 83.5654538545, pressure: 28.5043137197, rain: 0.0, rainRate: 0.0, usUnits: 1 LOOP: 2021-08-12 14:58:06 EDT (1628794686) active: 1.0, dateTime: 1628794686.48, lightning_distance: 7.99999999694, maxSolarRad: None, rainRate: 0, usUnits: 1 LOOP: 2021-08-12 14:58:30 EDT (1628794710) active: 1.0, dateTime: 1628794710.6, lightning_distance: 7.99999999694, maxSolarRad: None, rainRate: 0, usUnits: 1 REC: 2021-08-12 15:00:00 EDT (1628794800) active: 0.911764705882, altimeter: 30.352863321, barometer: 30.248274916, cloudbase: 4781.8918872, dateTime: 1628794800, dewpoint: 71.158468132, ET: None, heatindex: 89.7957954486, humidex: 100.932534378, interval: 5, lightning_distance: 7.99999999694, lightning_strike_count: 0.0, maxSolarRad: None, outHumidity: 64.3235294118, outTemp: 84.5603924356, pressure: 28.4987883408, rain: 0.0, rainRate: 0.0, usUnits: 1 LOOP: 2021-08-12 15:50:46 EDT (1628797846) active: 1.0, dateTime: 1628797846.38, lightning_distance: 7.99999999694, maxSolarRad: None, rainRate: 0, usUnits: 1 REC: 2021-08-12 15:55:00 EDT (1628798100) active: 1.0, altimeter: 30.3401020979, barometer: 30.2347797117, cloudbase: 4909.75354319, dateTime: 1628798100, dewpoint: 71.8556696944, ET: None, heatindex: 91.9474302989, humidex: 102.838512165, interval: 5, lightning_distance: 7.99999999694, lightning_strike_count: 0.0, maxSolarRad: None, outHumidity: 63.2432432432, outTemp: 85.8181836364, pressure: 28.4866625997, rain: 0.0, rainRate: 0.0, usUnits: 1 On Wednesday, August 11, 2021 at 3:12:22 PM UTC-4 Gene LeSage wrote: > Bingo, fixed by installing v2.1.0-RC MQTTSubscribeDriver. Adding > collect_observations = True and single_queue = True to [[topics], I get > desired behavior. Null lightning distance is recorded in absence of > strikes and with an test simulated lightning strike by battery charger > shorting, I get lightning strike recording and a lightning distance. Loop > shows both lightning distance and strikes in same loop: > > LOOP: 2021-08-11 14:56:07 EDT (1628708167) active: 1.0, cloudbase: > 5636.94172822, dateTime: 1628708167.42, dewpoint: 70.6358563958, heatindex: > 93.4196227576, humidex: 103.698544022, lightning_distance: None, > lightning_strike_count: 0, maxSolarRad: None, outHumidity: 57.0, outTemp: > 87.8, rainRate: 0, usUnits: 1 > LOOP: 2021-08-11 14:56:07 EDT (1628708167) active: 1.0, cloudbase: > 5636.94172822, dateTime: 1628708167.43, dewpoint: 70.6358563958, heatindex: > 93.4196227576, humidex: 103.698544022, lightning_distance: None, > lightning_strike_count: 0, maxSolarRad: None, outHumidity: 57.0, outTemp: > 87.8, rainRate: 0, usUnits: 1 > > Supporting the idea that lightning strikes and distances were being > interpreted at different times I caught two lightning strikes this morning > (before installing v2.1.0-RC) which shows the 8 am lightning strike before > lightning distance change and the 11 am lightning strike after lightning > distance change. By not being synchronized, the correction > (lightning_distance = lightning_distance if lightning_strike_count > 0 else > None) would never work. > > Thanks > > > > > > [image: Screenshot (14).png] > > On Wednesday, August 11, 2021 at 11:41:31 AM UTC-4 [email protected] > wrote: > >> I fear the root cause is that MQTTSubscribeDriver takes each MQTT message >> and creates a loop packet. This works ok for 'aggregated' messages like >> json or keyword (the original implementations). But for 'individual' >> payloads that are dependent on each other, it is problematic.I've already >> had to deal with this for wind data. MQTTSubscribe collects speed and >> direction data into a single loop packet. >> >> Since then I've been experimenting with collecting data and creating the >> loop packet from the collection. Looking at your loop packets, I think it >> might help you. >> >> The code is a pre-release which can be found here, >> https://github.com/bellrichm/WeeWX-MQTTSubscribe/releases/tag/v2.1.0-rc02 >> >> You will need to add two undocumented configuration options. >> >> [MQTTSubscribeDriver] >> . >> . >> . >> >> [[topics] >> # With the exception of wind data, by default a packet is created >> for every MQTT message received. >> # When this is true, MQTTSubscribe attempts to collect >> observations across messages into a packet. >> # Default is False. >> # This is experimental and may be removed. >> collect_observations = True >> >> # With the exception of wind data, by default a queue is created >> for every MQTT topic. >> # When this is true, MQTTSubsribe uses a single queue for all non >> wind data. >> # This is useful when 'collect_observations = True'. >> # Default is False. >> # This is experimental and may be removed. >> single_queue = True >> >> . >> . >> . >> >> With this, the incoming data should be collected and a loop packet only >> created when the arriving observation is already in the collection. Since, >> hopefully, the loop packet now has both strike and distance data, your >> [[Corrections]] should work. >> >> If this doesn't work. I'll have to go back to the drawing board and look >> for a different solution. If it comes to this, I'd need a debug level log >> for at least one archive period so that I can analyze the data. >> >> This is beta code, so the usual caveats apply. >> rich >> >> ps. >> If you are using rtl_433 to publish to MQTT, another option might be to >> use the 'events' topic that it publishes to. This is a json formatted >> message so the loop packet created for each MQTT message should have both >> the strike and distance data. >> >> On Wednesday, 11 August 2021 at 09:52:39 UTC-4 [email protected] wrote: >> >>> I am using weewx-MQTTSubscribe to bring data into Weewx from sensors, so >>> i don't use SDR sensor map, see config for mqttsubscribe here >>> https://github.com/bellrichm/WeeWX-MQTTSubscribe/wiki/Configuring A >>> number of unsuccessful tries to modify mqttsubscribe are commented out. >>> [MQTTSubscribeDriver] >>> [[topics]] >>> [[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/strike_count]]] >>> >>> [[[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/strike_count]]]] >>> name = lightning_strike_count #strikes_total >>> #strike_count >>> contains_total = true >>> # conversion_type = int >>> # units = count >>> # expires_after = 0 >>> >>> [[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/storm_dist]]] >>> >>> [[[[rtl_433/9b13b3f4-rtl433/devices/Acurite-6045M/C/223/storm_dist]]]] >>> name = lightning_distance #strike_dist #LS_1_distance >>> # units = mile >>> # conversion_type = int >>> # expires_after = 0 >>> >>> >>> >>> [[Corrections]] >>> # For each type, an arbitrary calibration expression can be >>> given. >>> # It should be in the units defined in the StdConvert section. >>> # Example: >>> lightning_distance = lightning_distance if >>> lightning_strike_count > 0 else None >>> >>> [Accumulator] >>> [[lightning_strike_count]] >>> extractor = sum >>> [[lightning_distance]] >>> extractor = min >>> # merger = minmax >>> >>> >>> On Wednesday, August 11, 2021 at 7:58:38 AM UTC-4 [email protected] >>> wrote: >>> >>>> I think going back to your original issue something seems to be off. >>>> Can you post you weewx.conf for lightning data? If you're using sdr, >>>> should >>>> contain the following in each section: >>>> >>>> [SDR] >>>> [[sensor_map]] >>>> ...... >>>> lightning_distance = distance.XXXX.AcuriteLightningPacket >>>> strikes_total = strikes_total.XXXX.AcuriteLightningPacket >>>> [[deltas]] >>>> rain = rain_total >>>> lightning_strike_count = strikes_total >>>> >>>> [StdCalibrate] >>>> >>>> [[Corrections]] >>>> lightning_distance = lightning_distance if >>>> lightning_strike_count > 0 else None >>>> >>>> [Accumulator] >>>> [[lightning_strike_count]] >>>> extractor = sum >>>> [[lightning_distance]] >>>> extractor = min >>>> >>>> On Tuesday, August 10, 2021 at 2:46:44 PM UTC-4 [email protected] wrote: >>>> >>>>> Initially I thought Rich's solution was going to work, adding his >>>>> suggested correction <lightning_distance = lightning_distance if >>>>> 'lightning_strike_count'in locals() and lightning_strike_count> 0 else >>>>> None> eliminated my fixed lightning_distance data in the absence of >>>>> lightning_counts, but it created a new problem, now I don't get >>>>> lightning_distance values with a lightning strike. I triggered the >>>>> lightning detector with a spark by transiently shorting a battery charger >>>>> at time 14:22:16 , which produced 2 detected lightnings and 14:22:24 >>>>> which >>>>> produced one, but the lightning_distance remains at None. My mqtt >>>>> explorer >>>>> reports the lightning detector is sending a constant 12-mile lightning >>>>> distant >>>>> >>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: 1628619736.82, >>>>> lightning_distance: None, maxSolarRad: None, outHumidity: 73.0, rainRate: >>>>> 0.32, usUnits: 1 >>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: 1628619736.81, >>>>> lightning_strike_count: 2.0, maxSolarRad: None, rainRate: 0.32, usUnits: >>>>> 1 >>>>> <-------------------------------------------------------- >>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: 1628619736.82, >>>>> lightning_distance: None, lightning_strike_count: 0.0, maxSolarRad: None, >>>>> rainRate: 0.32, usUnits: 1 >>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: 1628619736.83, >>>>> lightning_distance: None, lightning_strike_count: 0.0, maxSolarRad: None, >>>>> rainRate: 0.32, usUnits: 1 >>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: 1628619736.81, >>>>> lightning_distance: None, maxSolarRad: None, rainRate: 0.32, usUnits: 1 >>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: 1628619736.82, >>>>> lightning_distance: None, maxSolarRad: None, rainRate: 0.32, usUnits: 1 >>>>> LOOP: 2021-08-10 14:22:16 EDT (1628619736) dateTime: 1628619736.83, >>>>> lightning_distance: None, maxSolarRad: None, rainRate: 0.32, usUnits: 1 >>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.67, >>>>> lightning_distance: None, maxSolarRad: None, outTemp: 82.000004, >>>>> rainRate: >>>>> 0.32, usUnits: 1 >>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.68, >>>>> lightning_distance: None, maxSolarRad: None, outTemp: 82.000004, >>>>> rainRate: >>>>> 0.32, usUnits: 1 >>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.69, >>>>> lightning_distance: None, maxSolarRad: None, outTemp: 82.000004, >>>>> rainRate: >>>>> 0.32, usUnits: 1 >>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.67, >>>>> lightning_distance: None, maxSolarRad: None, outHumidity: 73.0, rainRate: >>>>> 0.32, usUnits: 1 >>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.68, >>>>> lightning_distance: None, maxSolarRad: None, outHumidity: 73.0, rainRate: >>>>> 0.32, usUnits: 1 >>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.69, >>>>> lightning_distance: None, maxSolarRad: None, outHumidity: 73.0, rainRate: >>>>> 0.32, usUnits: 1 >>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.68, >>>>> lightning_strike_count: 1.0, maxSolarRad: None, rainRate: 0.32, usUnits: >>>>> 1 >>>>> <----------------------------------------------------------- >>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.69, >>>>> lightning_distance: None, lightning_strike_count: 0.0, maxSolarRad: None, >>>>> rainRate: 0.32, usUnits: 1 >>>>> LOOP: 2021-08-10 14:22:24 EDT (1628619744) dateTime: 1628619744.69, >>>>> lightning_distance: None, lightning_strike_count: 0.0, maxSolarRad: None, >>>>> rainRate: 0.32, usUnits: 1 >>>>> >>>>> >>>>> On Tuesday, August 10, 2021 at 9:20:17 AM UTC-4 [email protected] >>>>> wrote: >>>>> >>>>>> I'm no python expert (probably know just enough to be dangerous), but >>>>>> something like this might get you want you want. >>>>>> >>>>>> # ligtning_strike_count must exist and have a count > 0 for >>>>>> lightning_distance to have a valid value >>>>>> lightning_distance = lightning_distance if 'lightning_strike_count'in >>>>>> locals() and lightning_strike_count> 0 else None >>>>>> >>>>>> rich >>>>>> On Tuesday, 10 August 2021 at 03:03:01 UTC-4 gjr80 wrote: >>>>>> >>>>>>> On Tuesday, 10 August 2021 at 08:37:42 UTC+10 [email protected] >>>>>>> wrote: >>>>>>> >>>>>>>> Here is my output running weewxd directly, I see three lines which >>>>>>>> show lightning_distance to be None, which is expected from >>>>>>>> corrections, but >>>>>>>> the following three lines show lightning_distance to be 11.999... It >>>>>>>> appears the correction " lightning_distance = lightning_distance if >>>>>>>> lightning_strike_count > 0 else None" is working but being ignored in >>>>>>>> final >>>>>>>> output to graph and database. Any additional thoughts? >>>>>>>> >>>>>>> >>>>>>> The reason you see no change to lightning_distance for some packets >>>>>>> is that for those packets lightning_strike_count does not exist so >>>>>>> your calibration expression fails and no change is made, in other words >>>>>>> lightning_distance remains as it was 11.9999999954. Why does this >>>>>>> happen, in layman's terms when lightning_strike_count is not in the >>>>>>> loop packet the calibration expression in effect has an unknown >>>>>>> variable ( >>>>>>> lightning_strike_count) and the calibration expression raises an >>>>>>> error. The StdCalibrate service which handles the calibration >>>>>>> expressions catches that error and discards that calibration >>>>>>> expression. >>>>>>> Given the limitations of the StdCalibrate service I am not aware of >>>>>>> any calibration expression that would do as you want, of course a WeeWX >>>>>>> and >>>>>>> python wizard might come along an prove me wrong! >>>>>>> >>>>>>> My thoughts on a solution, unless there is a simple solution in your >>>>>>> WeeWX driver or elsewhere upstream of WeeWX I would be writing your own >>>>>>> WeeWX service to look at the packet and make the necessary correction. >>>>>>> This >>>>>>> way you can use a bit of python code to do exactly what you want and >>>>>>> you >>>>>>> won't be limited by the single line expressions as used by >>>>>>> StdCalibrate. All up it should take no more than 10 lines of code. >>>>>>> The service could be a data_service or process_service (refer >>>>>>> weewx.conf [Engine] [[Services]]) but would need to appear before >>>>>>> StdArchive is called. The advantage of your own service is that >>>>>>> there is no need to change any one else's code (eg WeeWX driver, >>>>>>> upstream >>>>>>> code) so you will not have your changes lost during an upgrade (WeeWX >>>>>>> or >>>>>>> otherwise). >>>>>>> >>>>>>> Gary >>>>>>> >>>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ad772be1-26a4-48d4-9ce1-ab50e2e9fce2n%40googlegroups.com.
