Implemented in commit ea9fe13 <https://github.com/weewx/weewx/commit/ea9fe134a242ddd0ea1df9e1c4aa4ddea1542ff6>
-tk On Sat, Sep 15, 2018 at 5:38 AM Thomas Keffer <[email protected]> wrote: > Thanks, Chris > > I worry that this approach doesn't cover the case where the "previous" > record has, say, a timestamp far in the future. This patch would cause all > of the remaining records to get thrown out, but the "future" record to be > kept. > > In any case, it's way better than what we have, which allows negative > intervals. > > -tk > > > > On Fri, Sep 14, 2018 at 3:53 PM Chris Manton <[email protected]> wrote: > >> I suppose if the archived packets are presented out of order there could >> be a negative interval... >> >> I have not seen this but the wmr200 is full of wonderful surprises. >> >> One potential drive-by fix is to drop all packets satisfying this failing >> condition...here is one untested fix: >> >> >> $ git diff >> diff --git a/bin/weewx/drivers/wmr200.py b/bin/weewx/drivers/wmr200.py >> index eaab5a14..62b743e0 100644 >> --- a/bin/weewx/drivers/wmr200.py >> +++ b/bin/weewx/drivers/wmr200.py >> @@ -1943,7 +1943,13 @@ class WMR200(weewx.drivers.AbstractDevice): >> timestamp_packet_interval = timestamp_packet_current \ >> - timestamp_packet_previous >> >> - if pkt.timestamp_record() > (timestamp_packet_previous >> + if timestamp_packet_interval < 0: >> + loginf(('genStartup() Discarding received archive' >> + ' record that presented out-of-order' >> + ' current_threshold:%s previous_timestamp:%s') >> + % >> (weeutil.weeutil.timestamp_to_string(pkt.timestamp_record()) >> + >> weeutil.weeutil.timestamp_to_string(timestamp_packet_previous))) >> + elif pkt.timestamp_record() > (timestamp_packet_previous >> + self._archive_threshold): >> loginf(('genStartup() Discarding received archive' >> ' record exceeding archive interval cnt:%d' >> >> On Friday, September 14, 2018 at 6:07:48 AM UTC-7, Thomas Keffer wrote: >>> >>> Good. >>> >>> Looking through the WMR200 driver code (I didn't write it), it appears >>> that a positive value for 'interval' is not guaranteed. I'll ask the author >>> to take a look at it. >>> >>> -tk >>> >>> On Fri, Sep 14, 2018 at 5:44 AM Rahul Bedi <[email protected]> wrote: >>> >>>> Yup found a record with -ve interval, changed that to 5 as thats what I >>>> am using. >>>> >>>> And problem fixed !! >>>> >>>> Thanks a lot. >>>> >>>> Not sure how this bad data entered into the db. >>>> >>>> Will be great if there is utility that finds such errors in the db and >>>> flags them. >>>> >>>> - Rahul >>>> >>>> On Fri, Sep 14, 2018 at 5:48 PM Thomas Keffer <[email protected]> wrote: >>>> >>>>> I think the thing to do is to figure out how a negative value got in >>>>> there, fix that, then rebuild the daily summaries. >>>>> >>>>> One possibility is that a record in your database has a negative value >>>>> for 'interval' --- the time length of an archive record. Let's check that. >>>>> >>>>> *sqlite3 /var/lib/weewx/weewx.sdb* >>>>> sqlite> *SELECT datetime(dateTime, 'unixepoch','localtime'), interval >>>>> FROM archive WHERE interval<0;* >>>>> >>>>> -tk >>>>> >>>>> On Fri, Sep 14, 2018 at 5:06 AM Rahul Bedi <[email protected]> wrote: >>>>> >>>>>> Here is what I get: >>>>>> >>>>>> 2018-08-02 00:00:00|2851473.0|86400 >>>>>> 2018-08-03 00:00:00|2139447.0|86400 >>>>>> 2018-08-04 00:00:00|2867211.0|86400 >>>>>> 2018-08-05 00:00:00|3306195.0|86400 >>>>>> 2018-08-06 00:00:00|3167394.0|86400 >>>>>> 2018-08-07 00:00:00|2924283.0|86400 >>>>>> 2018-08-08 00:00:00|2255208.0|86400 >>>>>> 2018-08-09 00:00:00|2797065.0|86400 >>>>>> 2018-08-10 00:00:00|2693187.0|86400 >>>>>> 2018-08-11 00:00:00|2981601.0|86400 >>>>>> 2018-08-12 00:00:00|3869136.0|86400 >>>>>> 2018-08-13 00:00:00|3632787.0|86400 >>>>>> 2018-08-14 00:00:00|2651829.0|86400 >>>>>> 2018-08-15 00:00:00|2104410.0|86400 >>>>>> 2018-08-16 00:00:00|3682674.0|86400 >>>>>> 2018-08-17 00:00:00|3499992.0|86400 >>>>>> 2018-08-18 00:00:00|3855531.0|86400 >>>>>> 2018-08-19 00:00:00|3875121.0|86400 >>>>>> 2018-08-20 00:00:00|3915861.0|86400 >>>>>> 2018-08-21 00:00:00|3523719.0|86400 >>>>>> 2018-08-22 00:00:00|1637909.95179334|85620 >>>>>> 2018-08-23 00:00:00|506702.215048778|25380 >>>>>> 2018-08-24 00:00:00|523285.18735593|101340 >>>>>> 2018-08-25 00:00:00|2047231.90482286|92280 >>>>>> 2018-08-26 00:00:00|2207680.59885907|85800 >>>>>> 2018-08-27 00:00:00|2112653.43729687|78000 >>>>>> 2018-08-28 00:00:00|1927695.38113441|85200 >>>>>> 2018-08-29 00:00:00|1696867.80614767|76740 >>>>>> 2018-08-30 00:00:00|1892572.6564943|85500 >>>>>> 2018-08-31 00:00:00|1669207.56530998|86100 >>>>>> 2018-09-01 00:00:00|1882876.90336366|85080 >>>>>> 2018-09-02 00:00:00|1552040.97205686|86100 >>>>>> 2018-09-03 00:00:00|1550916.37957731|86100 >>>>>> 2018-09-04 00:00:00|1093704.34743737|85740 >>>>>> 2018-09-05 00:00:00|1688380.4274756|86100 >>>>>> 2018-09-06 00:00:00|2349485.94738889|86100 >>>>>> 2018-09-07 00:00:00|2676307.36370007|85560 >>>>>> 2018-09-08 00:00:00|2198034.86178985|86100 >>>>>> 2018-09-09 00:00:00|-7107216.29856356|-302280 >>>>>> 2018-09-10 00:00:00|288713.672694377|85560 >>>>>> 2018-09-11 00:00:00|351094.880509644|85860 >>>>>> 2018-09-12 00:00:00|441249.222795585|85500 >>>>>> 2018-09-13 00:00:00|60033.1634844758|85620 >>>>>> 2018-09-14 00:00:00|21222.2555021576|62820 >>>>>> >>>>>> I am guessing that the negative values above may be the culprit. How >>>>>> are these populated in the archive_day_wind table or should I just edit >>>>>> these values out ? >>>>>> >>>>>> - Rahul >>>>>> >>>>>> >>>>>> On Fri, Sep 14, 2018 at 5:19 PM Thomas Keffer <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> You're on the right track: there must be an bad value in your >>>>>>> database. Did you, by any chance, use a visual sql editor? They often >>>>>>> put >>>>>>> null strings in the database. >>>>>>> >>>>>>> In any case, the field 'wsquaresum' and 'sumtime' are used by rms. >>>>>>> So, try this: >>>>>>> >>>>>>> *sqlite3 /var/lib/weewx/weewx.sdb* >>>>>>> sqlite> *SELECT datetime(dateTime, 'unixepoch','localtime'), >>>>>>> wsquaresum, sumtime FROM archive_day_wind WHERE dateTime > 1533081600;* >>>>>>> >>>>>>> This will show all values from (roughly) 1-August (about 45 values). >>>>>>> If the bad value is before that, we'll have to use a different strategy, >>>>>>> rather than go looking for a needle in a haystack. >>>>>>> >>>>>>> -tk >>>>>>> >>>>>>> On Thu, Sep 13, 2018 at 11:35 PM Rahul Bedi <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> I found rms used in the statictics.ico and hilo.ico files in the >>>>>>>> Seasons directory. removed those segments. Diff as follows: >>>>>>>> >>>>>>>> $ diff statistics.14092018.inc statistics.inc >>>>>>>> >>>>>>>> 134,140d133 >>>>>>>> < <tr> >>>>>>>> < <td class="label">$obs.label.wind_rms</td> >>>>>>>> < <td class="units">$unit.label.wind</td> >>>>>>>> < #for $archive in $archive_data >>>>>>>> < <td class="data >>>>>>>> new_row">$archive.wind.rms.formatted</td> >>>>>>>> < #end for >>>>>>>> < </tr> >>>>>>>> >>>>>>>> $ diff hilo.inc hilo.14092018.inc >>>>>>>> >>>>>>>> 132a133,140 >>>>>>>> > <tr> >>>>>>>> > <td class="label">$obs.label.wind_rms</td> >>>>>>>> > #for $archive in $archive_data >>>>>>>> > <td class="data new_row hilo_$archive[0]"> >>>>>>>> > $archive[1].wind.rms.formatted</td> >>>>>>>> > #end for >>>>>>>> > <td class="units">$unit.label.wind</td> >>>>>>>> > </tr> >>>>>>>> >>>>>>>> Apart from this any file referring to rms were exactly the same as >>>>>>>> the original install. Attaching skin.conf with this message. >>>>>>>> >>>>>>>> As expected the problem is gone but rms is not getting reported. >>>>>>>> Which means the issue still stays. >>>>>>>> >>>>>>>> Could you help me with what fields in the archive are referred to >>>>>>>> by rms ? I can try to manually check for any value discrepancy. >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> - Rahul >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Sep 14, 2018 at 9:14 AM Rahul Bedi <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Thanks Gary. >>>>>>>>> >>>>>>>>> Charges were done about a month ago but have seen no errors till >>>>>>>>> this episode. I will do a diff with the original skin files and post >>>>>>>>> it >>>>>>>>> here. >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> - Rahul >>>>>>>>> >>>>>>>>> On Fri 14 Sep, 2018, 7:35 AM gjr80, <[email protected]> wrote: >>>>>>>>> >>>>>>>>>> On Friday, 14 September 2018 11:30:42 UTC+10, Rahul Bedi wrote: >>>>>>>>>>> >>>>>>>>>>> As I have made no code or config changes since the day before, I >>>>>>>>>>> believe it doing this for the right data. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This sounds to me like you have changed some of the seasons skin >>>>>>>>>> files. The error is coming form the use of a .rms tag in one of >>>>>>>>>> the seasons .tmpl or .inc files. The seasons skin only uses the >>>>>>>>>> .rms tag with the WeeWX wind field. As Tom indicated .rms only >>>>>>>>>> works against the wind field, if your changes have used the .rms >>>>>>>>>> tag against another field then this could be the cause. I suggest >>>>>>>>>> you go >>>>>>>>>> through all of your changes to the seasons skin files and see what >>>>>>>>>> fields >>>>>>>>>> are being used with the .rms tag. Until we know for sure, and >>>>>>>>>> given you seem to have made changes to the skin, we will likely be >>>>>>>>>> chasing >>>>>>>>>> our tails. >>>>>>>>>> >>>>>>>>>> Gary >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to a topic >>>>>>>>>> in the Google Groups "weewx-user" group. >>>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>>> https://groups.google.com/d/topic/weewx-user/LWmSpcVfdmo/unsubscribe >>>>>>>>>> . >>>>>>>>>> To unsubscribe from this group and all its topics, send an email >>>>>>>>>> to [email protected]. >>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>> >>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "weewx-user" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to a topic in >>>>>>> the Google Groups "weewx-user" group. >>>>>>> To unsubscribe from this topic, visit >>>>>>> https://groups.google.com/d/topic/weewx-user/LWmSpcVfdmo/unsubscribe >>>>>>> . >>>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>>> [email protected]. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "weewx-user" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to a topic in the >>>>> Google Groups "weewx-user" group. >>>>> To unsubscribe from this topic, visit >>>>> https://groups.google.com/d/topic/weewx-user/LWmSpcVfdmo/unsubscribe. >>>>> To unsubscribe from this group and all its topics, send an email to >>>>> [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "weewx-user" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
