Thanks, Chris I worry that this approach doesn't cover the case where the "previous" record has, say, a timestamp far in the future. This patch would cause all of the remaining records to get thrown out, but the "future" record to be kept.
In any case, it's way better than what we have, which allows negative intervals. -tk On Fri, Sep 14, 2018 at 3:53 PM Chris Manton <[email protected]> wrote: > I suppose if the archived packets are presented out of order there could > be a negative interval... > > I have not seen this but the wmr200 is full of wonderful surprises. > > One potential drive-by fix is to drop all packets satisfying this failing > condition...here is one untested fix: > > > $ git diff > diff --git a/bin/weewx/drivers/wmr200.py b/bin/weewx/drivers/wmr200.py > index eaab5a14..62b743e0 100644 > --- a/bin/weewx/drivers/wmr200.py > +++ b/bin/weewx/drivers/wmr200.py > @@ -1943,7 +1943,13 @@ class WMR200(weewx.drivers.AbstractDevice): > timestamp_packet_interval = timestamp_packet_current \ > - timestamp_packet_previous > > - if pkt.timestamp_record() > (timestamp_packet_previous > + if timestamp_packet_interval < 0: > + loginf(('genStartup() Discarding received archive' > + ' record that presented out-of-order' > + ' current_threshold:%s previous_timestamp:%s') > + % > (weeutil.weeutil.timestamp_to_string(pkt.timestamp_record()) > + > weeutil.weeutil.timestamp_to_string(timestamp_packet_previous))) > + elif pkt.timestamp_record() > (timestamp_packet_previous > + self._archive_threshold): > loginf(('genStartup() Discarding received archive' > ' record exceeding archive interval cnt:%d' > > On Friday, September 14, 2018 at 6:07:48 AM UTC-7, Thomas Keffer wrote: >> >> Good. >> >> Looking through the WMR200 driver code (I didn't write it), it appears >> that a positive value for 'interval' is not guaranteed. I'll ask the author >> to take a look at it. >> >> -tk >> >> On Fri, Sep 14, 2018 at 5:44 AM Rahul Bedi <[email protected]> wrote: >> >>> Yup found a record with -ve interval, changed that to 5 as thats what I >>> am using. >>> >>> And problem fixed !! >>> >>> Thanks a lot. >>> >>> Not sure how this bad data entered into the db. >>> >>> Will be great if there is utility that finds such errors in the db and >>> flags them. >>> >>> - Rahul >>> >>> On Fri, Sep 14, 2018 at 5:48 PM Thomas Keffer <[email protected]> wrote: >>> >>>> I think the thing to do is to figure out how a negative value got in >>>> there, fix that, then rebuild the daily summaries. >>>> >>>> One possibility is that a record in your database has a negative value >>>> for 'interval' --- the time length of an archive record. Let's check that. >>>> >>>> *sqlite3 /var/lib/weewx/weewx.sdb* >>>> sqlite> *SELECT datetime(dateTime, 'unixepoch','localtime'), interval >>>> FROM archive WHERE interval<0;* >>>> >>>> -tk >>>> >>>> On Fri, Sep 14, 2018 at 5:06 AM Rahul Bedi <[email protected]> wrote: >>>> >>>>> Here is what I get: >>>>> >>>>> 2018-08-02 00:00:00|2851473.0|86400 >>>>> 2018-08-03 00:00:00|2139447.0|86400 >>>>> 2018-08-04 00:00:00|2867211.0|86400 >>>>> 2018-08-05 00:00:00|3306195.0|86400 >>>>> 2018-08-06 00:00:00|3167394.0|86400 >>>>> 2018-08-07 00:00:00|2924283.0|86400 >>>>> 2018-08-08 00:00:00|2255208.0|86400 >>>>> 2018-08-09 00:00:00|2797065.0|86400 >>>>> 2018-08-10 00:00:00|2693187.0|86400 >>>>> 2018-08-11 00:00:00|2981601.0|86400 >>>>> 2018-08-12 00:00:00|3869136.0|86400 >>>>> 2018-08-13 00:00:00|3632787.0|86400 >>>>> 2018-08-14 00:00:00|2651829.0|86400 >>>>> 2018-08-15 00:00:00|2104410.0|86400 >>>>> 2018-08-16 00:00:00|3682674.0|86400 >>>>> 2018-08-17 00:00:00|3499992.0|86400 >>>>> 2018-08-18 00:00:00|3855531.0|86400 >>>>> 2018-08-19 00:00:00|3875121.0|86400 >>>>> 2018-08-20 00:00:00|3915861.0|86400 >>>>> 2018-08-21 00:00:00|3523719.0|86400 >>>>> 2018-08-22 00:00:00|1637909.95179334|85620 >>>>> 2018-08-23 00:00:00|506702.215048778|25380 >>>>> 2018-08-24 00:00:00|523285.18735593|101340 >>>>> 2018-08-25 00:00:00|2047231.90482286|92280 >>>>> 2018-08-26 00:00:00|2207680.59885907|85800 >>>>> 2018-08-27 00:00:00|2112653.43729687|78000 >>>>> 2018-08-28 00:00:00|1927695.38113441|85200 >>>>> 2018-08-29 00:00:00|1696867.80614767|76740 >>>>> 2018-08-30 00:00:00|1892572.6564943|85500 >>>>> 2018-08-31 00:00:00|1669207.56530998|86100 >>>>> 2018-09-01 00:00:00|1882876.90336366|85080 >>>>> 2018-09-02 00:00:00|1552040.97205686|86100 >>>>> 2018-09-03 00:00:00|1550916.37957731|86100 >>>>> 2018-09-04 00:00:00|1093704.34743737|85740 >>>>> 2018-09-05 00:00:00|1688380.4274756|86100 >>>>> 2018-09-06 00:00:00|2349485.94738889|86100 >>>>> 2018-09-07 00:00:00|2676307.36370007|85560 >>>>> 2018-09-08 00:00:00|2198034.86178985|86100 >>>>> 2018-09-09 00:00:00|-7107216.29856356|-302280 >>>>> 2018-09-10 00:00:00|288713.672694377|85560 >>>>> 2018-09-11 00:00:00|351094.880509644|85860 >>>>> 2018-09-12 00:00:00|441249.222795585|85500 >>>>> 2018-09-13 00:00:00|60033.1634844758|85620 >>>>> 2018-09-14 00:00:00|21222.2555021576|62820 >>>>> >>>>> I am guessing that the negative values above may be the culprit. How >>>>> are these populated in the archive_day_wind table or should I just edit >>>>> these values out ? >>>>> >>>>> - Rahul >>>>> >>>>> >>>>> On Fri, Sep 14, 2018 at 5:19 PM Thomas Keffer <[email protected]> >>>>> wrote: >>>>> >>>>>> You're on the right track: there must be an bad value in your >>>>>> database. Did you, by any chance, use a visual sql editor? They often put >>>>>> null strings in the database. >>>>>> >>>>>> In any case, the field 'wsquaresum' and 'sumtime' are used by rms. >>>>>> So, try this: >>>>>> >>>>>> *sqlite3 /var/lib/weewx/weewx.sdb* >>>>>> sqlite> *SELECT datetime(dateTime, 'unixepoch','localtime'), >>>>>> wsquaresum, sumtime FROM archive_day_wind WHERE dateTime > 1533081600;* >>>>>> >>>>>> This will show all values from (roughly) 1-August (about 45 values). >>>>>> If the bad value is before that, we'll have to use a different strategy, >>>>>> rather than go looking for a needle in a haystack. >>>>>> >>>>>> -tk >>>>>> >>>>>> On Thu, Sep 13, 2018 at 11:35 PM Rahul Bedi <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> I found rms used in the statictics.ico and hilo.ico files in the >>>>>>> Seasons directory. removed those segments. Diff as follows: >>>>>>> >>>>>>> $ diff statistics.14092018.inc statistics.inc >>>>>>> >>>>>>> 134,140d133 >>>>>>> < <tr> >>>>>>> < <td class="label">$obs.label.wind_rms</td> >>>>>>> < <td class="units">$unit.label.wind</td> >>>>>>> < #for $archive in $archive_data >>>>>>> < <td class="data new_row">$archive.wind.rms.formatted</td> >>>>>>> < #end for >>>>>>> < </tr> >>>>>>> >>>>>>> $ diff hilo.inc hilo.14092018.inc >>>>>>> >>>>>>> 132a133,140 >>>>>>> > <tr> >>>>>>> > <td class="label">$obs.label.wind_rms</td> >>>>>>> > #for $archive in $archive_data >>>>>>> > <td class="data new_row hilo_$archive[0]"> >>>>>>> > $archive[1].wind.rms.formatted</td> >>>>>>> > #end for >>>>>>> > <td class="units">$unit.label.wind</td> >>>>>>> > </tr> >>>>>>> >>>>>>> Apart from this any file referring to rms were exactly the same as >>>>>>> the original install. Attaching skin.conf with this message. >>>>>>> >>>>>>> As expected the problem is gone but rms is not getting reported. >>>>>>> Which means the issue still stays. >>>>>>> >>>>>>> Could you help me with what fields in the archive are referred to by >>>>>>> rms ? I can try to manually check for any value discrepancy. >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> - Rahul >>>>>>> >>>>>>> >>>>>>> On Fri, Sep 14, 2018 at 9:14 AM Rahul Bedi <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Thanks Gary. >>>>>>>> >>>>>>>> Charges were done about a month ago but have seen no errors till >>>>>>>> this episode. I will do a diff with the original skin files and post >>>>>>>> it >>>>>>>> here. >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> - Rahul >>>>>>>> >>>>>>>> On Fri 14 Sep, 2018, 7:35 AM gjr80, <[email protected]> wrote: >>>>>>>> >>>>>>>>> On Friday, 14 September 2018 11:30:42 UTC+10, Rahul Bedi wrote: >>>>>>>>>> >>>>>>>>>> As I have made no code or config changes since the day before, I >>>>>>>>>> believe it doing this for the right data. >>>>>>>>>> >>>>>>>>> >>>>>>>>> This sounds to me like you have changed some of the seasons skin >>>>>>>>> files. The error is coming form the use of a .rms tag in one of >>>>>>>>> the seasons .tmpl or .inc files. The seasons skin only uses the >>>>>>>>> .rms tag with the WeeWX wind field. As Tom indicated .rms only >>>>>>>>> works against the wind field, if your changes have used the .rms >>>>>>>>> tag against another field then this could be the cause. I suggest you >>>>>>>>> go >>>>>>>>> through all of your changes to the seasons skin files and see what >>>>>>>>> fields >>>>>>>>> are being used with the .rms tag. Until we know for sure, and >>>>>>>>> given you seem to have made changes to the skin, we will likely be >>>>>>>>> chasing >>>>>>>>> our tails. >>>>>>>>> >>>>>>>>> Gary >>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to a topic in >>>>>>>>> the Google Groups "weewx-user" group. >>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>> https://groups.google.com/d/topic/weewx-user/LWmSpcVfdmo/unsubscribe >>>>>>>>> . >>>>>>>>> To unsubscribe from this group and all its topics, send an email >>>>>>>>> to [email protected]. >>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "weewx-user" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to a topic in >>>>>> the Google Groups "weewx-user" group. >>>>>> To unsubscribe from this topic, visit >>>>>> https://groups.google.com/d/topic/weewx-user/LWmSpcVfdmo/unsubscribe. >>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>> [email protected]. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "weewx-user" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/weewx-user/LWmSpcVfdmo/unsubscribe. >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
