I'm sure everyone is tired of me and this topic but I'm trying to understand what is happening here.
I have switched to using " $day.co2.last" which should work for all hourly data sent except for maybe something very close to midnight. Looking at the contents of weewx.sdb I still notice gaps in the db. With weewx.log, I can see that a couple of missing data were sent correctly but: Dec 9 12:08:11 n4mrv weewx[448959] INFO user.MQTTSubscribe: (Service) TopicManager ignoring record outside of interval 1639069690.000000 1639069692.000000 1639069689.979975 co2: 423.0, dateTime: 1639069689.9799752, usUnits: 1 Dec 9 13:09:25 n4mrv weewx[448959] INFO user.MQTTSubscribe: (Service) TopicManager ignoring record outside of interval 1639073364.000000 1639073366.000000 1639073363.761206 co2: 415.0, dateTime: 1639073363.7612064, usUnits: 1 This happens for a couple of hours and the data is not included in the db. On the third hour, the data is sent and correctly put into the db Dec 9 14:10:37 n4mrv weewx[448959] DEBUG user.MQTTSubscribe: (Service) TopicManager data-> outgoing accumulated mytopic: co2: 400.0, dateTime: 1639077038.0, usUnits: 1 The attached file shows what is happening. Thanks in advance for any suggestion where I'm going wrong here. There may be something I have incorrect in MQTTSubscribe. I'll be happy to send more information if needed. Bob On Monday, December 6, 2021 at 8:11:50 PM UTC-5 [email protected] wrote: > No reason why you can't use $day.co2.last. It will give you the last valid > value for the day. > > On Mon, Dec 6, 2021 at 7:38 AM [email protected] <[email protected]> wrote: > >> Ok, I seem to have jumped the gun :( >> >> Things are much better but I'm still getting the N/A in place of data >> every few archive records (current data). My suspicion here is that the >> sensor is not sending on exactly 60 minute intervals. Looking at the actual >> times in the LoRa server, it's 60, 61 or 62 minute intervals--not always >> the same. Is it possible, using the basic " $hour.co2.last" syntax, to >> change the "hour" to say 65 minutes? This would probably catch the changes >> from one hour. I have attached a file of what is in the weewx.sdb showing >> the missing data in certain instances. If the sensor has updated just after >> the archive time (15 minutes) then it misses recording new data. >> I'm just guessing here at what might be going on. The graph seems to be >> picking up the data with no problem. >> Thanks. >> Bob[image: zbsng7lr.bmp] >> >> On Sunday, December 5, 2021 at 12:41:03 PM UTC-5 [email protected] wrote: >> >>> It looks like " $hour.co2.last" did it, at least the last four hours are >>> updating correctly with no N/As mixed in. >>> >>> Many thanks to all who contributed help on this, and much appreciated is >>> all the hard work that goes into weewx maintenance and updates by many! >>> Bob >>> http://grattans.org/wx >>> >>> On Sunday, December 5, 2021 at 8:10:12 AM UTC-5 [email protected] wrote: >>> >>>> I'd go with Tom's advice, but if you want to get the cache working I'd >>>> need a log file with debug set to 2. It needs to have a few archive >>>> records >>>> before the first CO2 reading arrives via MQTT and a few records after. >>>> Warning, the file is going to large, very large. >>>> rich >>>> >>>> On Saturday, 4 December 2021 at 20:08:19 UTC-5 [email protected] wrote: >>>> >>>>> Take the accumulator stuff out. >>>>> >>>>> Why not just use $hour.co2.last? >>>>> >>>>> >>>>> >>>>> On Sat, Dec 4, 2021 at 4:17 PM gjr80 <[email protected]> wrote: >>>>> >>>>>> Suggesting use of $hour_delta=1 wasn’t wise on my part as data >>>>>> arriving once per 60 minutes will likely lead to some occasions where no >>>>>> data is displayed depending on exactly when the data arrives. >>>>>> Conceivably >>>>>> data arriving or being processed a few seconds late could fall outside >>>>>> of >>>>>> $hour_delta=1. >>>>>> >>>>>> Actually, thinking some more, you my be able to use the aggregate >>>>>> ‘last’ to get the last value in the aggregation period, so try >>>>>> $span($hour_delta=2).co2.last. I may be mistaken but I think the >>>>>> aggregates >>>>>> exclude the None values so this tag may give you the most recent >>>>>> non-None >>>>>> value in the last two hours. Refer to >>>>>> http://weewx.com/docs/customizing.htm#Tag_$span and >>>>>> http://weewx.com/docs/customizing.htm#aggregation_types. >>>>>> >>>>>> Failing that you may like to try $span($time_delta=3900).co2.avg or >>>>>> 4200 to display the average over the last 70 or 80 minutes. Be aware >>>>>> though that just as $hour_delta=1 missed some records going the other >>>>>> way >>>>>> to > 1 hour will likely see more than one record be included in some >>>>>> aggregates, probably not an issue but something to be aware of. >>>>>> >>>>>> Gary >>>>>> On Sunday, 5 December 2021 at 08:35:01 UTC+10 [email protected] wrote: >>>>>> >>>>>>> I've tried a few suggestions. The sensor output time can be changed >>>>>>> to a shorter period but, at this point, I prefer to keep it at 60 >>>>>>> minutes >>>>>>> since the internal battery looks impossible to change easily. At 60 >>>>>>> minutes >>>>>>> it is supposed to last a little over 3 years. >>>>>>> >>>>>>> I added the suggestion " $span($hour_delta=1).co2.avg" which seems >>>>>>> to bring in data more often but there are still some periods of "N/A." >>>>>>> >>>>>>> I added " expires_after = none" but for now that doesn't seem to do >>>>>>> anything. >>>>>>> >>>>>>> The suggestion of writing data to a scratch file may be the only way >>>>>>> to go if I can't configure it another way. >>>>>>> >>>>>>> I have produced a daily graph on the main page >>>>>>> >>>>>>> [image: yafe6kzk.bmp] >>>>>>> >>>>>>> The sensor is in the house for now and the peaks seem to come from >>>>>>> using the oven and burners on the gas stove. >>>>>>> >>>>>>> Thanks for all the help. I'll give this a day or so to accumulate >>>>>>> some data. >>>>>>> On Saturday, December 4, 2021 at 4:46:26 PM UTC-5 [email protected] >>>>>>> wrote: >>>>>>> >>>>>>>> Bob, >>>>>>>> As noted earlier, now that you have a base MQTTSubscribe >>>>>>>> configuration up and running, you probably want to look at its >>>>>>>> 'archive >>>>>>>> record cache capability'. Details are here, >>>>>>>> >>>>>>>> https://github.com/bellrichm/WeeWX-MQTTSubscribe/wiki/Configuring-additional-options#expires_after >>>>>>>> >>>>>>>> I think that setting expires_after to none would be the simplest. >>>>>>>> So you would end up with something like this. >>>>>>>> [MQTTSubscribeService] >>>>>>>> [[topics]] >>>>>>>> [[[mytopic]]] >>>>>>>> ignore = true >>>>>>>> [[[[CO2_Value_PPM]]]] >>>>>>>> name = co2 >>>>>>>> ignore = false >>>>>>>> expires_after = none >>>>>>>> >>>>>>>> rich >>>>>>>> >>>>>>>> On Saturday, 4 December 2021 at 15:09:39 UTC-5 gjr80 wrote: >>>>>>>> >>>>>>>>> The other approach is to give WeeWX at least one co2 value per >>>>>>>>> archive period (it can be the same value) but that is a function of >>>>>>>>> your >>>>>>>>> data source and the driver being used and I don’t know the >>>>>>>>> limitations/capabilities of each. >>>>>>>>> >>>>>>>>> Gary >>>>>>>>> >>>>>>>>> On Sunday, 5 December 2021 at 05:56:10 UTC+10 gjr80 wrote: >>>>>>>>> >>>>>>>>>> The issue here is that the ‘co2 field’ receives data only once >>>>>>>>>> per hour, so based on a 15 minute archive period only one in four >>>>>>>>>> archive >>>>>>>>>> records will have co2 data. Changing the accumulator type and >>>>>>>>>> extraction >>>>>>>>>> policy will not change this. I would leave the co2 accumulator >>>>>>>>>> settings out >>>>>>>>>> of weewx.conf. I doubt whether $last will be much help either; $last >>>>>>>>>> displays data from the last known good record in the database as >>>>>>>>>> opposed to >>>>>>>>>> the current record used by $current. If there is no co2 data in >>>>>>>>>> most >>>>>>>>>> records then $last will provide a similar result. >>>>>>>>>> >>>>>>>>>> You may have better luck with some sort of aggregate over the >>>>>>>>>> past hour, say something like (untested) >>>>>>>>>> $span($hour_delta=1).co2.avg. >>>>>>>>>> The actual aggregate is probably meaningless since you only have one >>>>>>>>>> value >>>>>>>>>> per hour so min be mx will work just as well. >>>>>>>>>> >>>>>>>>>> Gary >>>>>>>>>> On Sunday, 5 December 2021 at 03:59:46 UTC+10 [email protected] >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Sorry, here is some info. This time, I re-inserted the >>>>>>>>>>> [Accumulator] paragraph at a different place in weewx.conf and it >>>>>>>>>>> restarted >>>>>>>>>>> without an exit. The web data is still showing "$last.co2" >>>>>>>>>>> It ran for a while and then stopped. Debug file attached. Thanks >>>>>>>>>>> weewx.conf >>>>>>>>>>> >>>>>>>>>>> # Options for 'MQTTSubscribeService' >>>>>>>>>>> [Accumulator] >>>>>>>>>>> [[co2]] >>>>>>>>>>> accumulator = firstlast >>>>>>>>>>> extractor = last >>>>>>>>>>> >>>>>>>>>>> [MQTTSubscribeService] >>>>>>>>>>> # This section is for the MQTTSubscribe service. >>>>>>>>>>> >>>>>>>>>>> # Turn the service on and off. >>>>>>>>>>> # Default is: true >>>>>>>>>>> # Only used by the service. >>>>>>>>>>> enable = true # false >>>>>>>>>>> ================================================== >>>>>>>>>>> index.html.tmpl >>>>>>>>>>> <tr class = "even"> >>>>>>>>>>> <td class="stats_label">105 Crawl >>>>>>>>>>> Temperature</td> >>>>>>>>>>> <td class="stats_data">$current.extraTemp1 / >>>>>>>>>>> $current.extraTemp1.degree_C</td> >>>>>>>>>>> </tr> >>>>>>>>>>> <tr class = "even"> >>>>>>>>>>> <td class="stats_label">CO2 level</td> >>>>>>>>>>> <td class="stats_data">$last.co2</td> >>>>>>>>>>> </tr> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ================================================== >>>>>>>>>>> >>>>>>>>>>> Standard web page reads: (http://grattans.org/wx) >>>>>>>>>>> >>>>>>>>>>> 105 Crawl Temperature 72.0°F / 22.2°C >>>>>>>>>>> CO2 level $last.co2 >>>>>>>>>>> >>>>>>>>>>> On Saturday, December 4, 2021 at 12:07:09 PM UTC-5 vince wrote: >>>>>>>>>>> >>>>>>>>>>>> On Saturday, December 4, 2021 at 7:16:00 AM UTC-8 >>>>>>>>>>>> [email protected] wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I tried changing $current.co2 to $last.co2 but it only printed >>>>>>>>>>>>> "$last.co2" on the web page. >>>>>>>>>>>>> >>>>>>>>>>>>> Adding the [Accumulator] section to weewx.conf only caused an >>>>>>>>>>>>> error and exit when I restarted weewx. I tried adding this in >>>>>>>>>>>>> several >>>>>>>>>>>>> places but none seems to let weewx restart without an exit error. >>>>>>>>>>>>> Is there >>>>>>>>>>>>> a special place to add this? Couldn't find anything in the doc >>>>>>>>>>>>> about >>>>>>>>>>>>> placement. Thanks >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> Usual answers apply. >>>>>>>>>>>> >>>>>>>>>>>> Unless you post the error messages verbatim and/or the changes >>>>>>>>>>>> you made to the skin, we are not going to be able to help you. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "weewx-user" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/weewx-user/60583ac3-73f3-479c-91d0-34fcdf1c0865n%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/weewx-user/60583ac3-73f3-479c-91d0-34fcdf1c0865n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/weewx-user/57504a16-555a-4123-a5dd-ae7b39861befn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/weewx-user/57504a16-555a-4123-a5dd-ae7b39861befn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/5bd52a4f-c8b4-42c7-bc0b-ce7e6d3ca4fen%40googlegroups.com.
Dec 9 11:06:56 n4mrv weewx[448959] DEBUG user.MQTTSubscribe: (Service) TopicManager data-> incoming mytopic: co2: 448.0 Dec 9 11:06:57 n4mrv weewx[448959] DEBUG user.MQTTSubscribe: (Service) TopicManager data-> outgoing mytopic: co2: 448.0, dateTime: 1639066016.1437914, usUnits: 1 Dec 9 11:06:57 n4mrv weewx[448959] DEBUG user.MQTTSubscribe: (Service) TopicManager data-> outgoing accumulated mytopic: co2: 448.0, dateTime: 1639066017.0, usUnits: 1 Dec 9 11:06:57 n4mrv weewx[448959] DEBUG user.MQTTSubscribe: (Service) data-> final packet is 2021-12-09 11:06:57 EST (1639066017): barometer: 30.311, co2: 448.0, consBatteryVoltage: 4.72, dateTime: 1639066017.0, dayET: 0.007, .... Dec 9 12:08:09 n4mrv weewx[448959] DEBUG user.MQTTSubscribe: (Service) TopicManager data-> incoming mytopic: co2: 423.0 Dec 9 12:08:11 n4mrv weewx[448959] DEBUG user.MQTTSubscribe: (Service) TopicManager data-> outgoing mytopic: co2: 423.0, dateTime: 1639069689.9799752, usUnits: 1 Dec 9 12:08:11 n4mrv weewx[448959] INFO user.MQTTSubscribe: (Service) TopicManager ignoring record outside of interval 1639069690.000000 1639069692.000000 1639069689.979975 co2: 423.0, dateTime: 1639069689.9799752, usUnits: 1 Dec 9 13:09:23 n4mrv weewx[448959] DEBUG user.MQTTSubscribe: (Service) TopicManager data-> incoming mytopic: co2: 415.0 Dec 9 13:09:25 n4mrv weewx[448959] DEBUG user.MQTTSubscribe: (Service) TopicManager data-> outgoing mytopic: co2: 415.0, dateTime: 1639073363.7612064, usUnits: 1 Dec 9 13:09:25 n4mrv weewx[448959] INFO user.MQTTSubscribe: (Service) TopicManager ignoring record outside of interval 1639073364.000000 1639073366.000000 1639073363.761206 co2: 415.0, dateTime: 1639073363.7612064, usUnits: 1 Dec 9 14:10:37 n4mrv weewx[448959] DEBUG user.MQTTSubscribe: (Service) TopicManager data-> incoming mytopic: co2: 400.0 Dec 9 14:10:37 n4mrv weewx[448959] DEBUG user.MQTTSubscribe: (Service) TopicManager data-> outgoing mytopic: co2: 400.0, dateTime: 1639077037.5615456, usUnits: 1 Dec 9 14:10:37 n4mrv weewx[448959] DEBUG user.MQTTSubscribe: (Service) TopicManager data-> outgoing accumulated mytopic: co2: 400.0, dateTime: 1639077038.0, usUnits: 1 sqlite> .open weewx.sdb sqlite> select datetime(dateTime, 'unixepoch', 'localtime'), co2 from archive where datetime(dateTime,'unixepoch','localtime') > "2021-12-07 09:00:00" ; 2021-12-09 09:15:00|479.0 2021-12-09 09:30:00| 2021-12-09 09:45:00| 2021-12-09 10:00:00| 2021-12-09 10:15:00|466.0 2021-12-09 10:30:00| 2021-12-09 10:45:00| 2021-12-09 11:00:00| 2021-12-09 11:15:00|448.0 data ok sent at 11:06 2021-12-09 11:30:00| 2021-12-09 11:45:00| 2021-12-09 12:00:00| 2021-12-09 12:15:00| missing data 423.0 sent 12:08 2021-12-09 12:30:00| 2021-12-09 12:45:00| 2021-12-09 13:00:00| 2021-12-09 13:15:00| missing data 415.0 sent 13:09 2021-12-09 13:30:00| 2021-12-09 13:45:00| 2021-12-09 14:00:00| 2021-12-09 14:15:00|400.0 data ok sent at 14:10
