Negative value. Thanks.
- Rahul On Sat 15 Sep, 2018, 7:55 PM Thomas Keffer, <[email protected]> wrote: > What's a "-ve" value? > > -tk > > On Sat, Sep 15, 2018 at 6:10 AM Rahul Bedi <[email protected]> wrote: > >> On the topics of negative numbers, I have also realized that in cases >> where I shutdown weewx running on Pi, unplug the USB cable running to the >> WMR200, plug it back and restart the weewx, sometimes I see -ve value for >> rain getting populated in the database which I have to manually remove. >> Looks like WMR200 is certainly an interesting WS. Is there a way to guard >> against such erroneous records ? >> >> Thanks. >> >> - Rahul >> >> On Sat, Sep 15, 2018 at 6:20 PM Thomas Keffer <[email protected]> wrote: >> >>> Implemented in commit ea9fe13 >>> <https://github.com/weewx/weewx/commit/ea9fe134a242ddd0ea1df9e1c4aa4ddea1542ff6> >>> >>> -tk >>> >>> On Sat, Sep 15, 2018 at 5:38 AM Thomas Keffer <[email protected]> wrote: >>> >>>> Thanks, Chris >>>> >>>> I worry that this approach doesn't cover the case where the "previous" >>>> record has, say, a timestamp far in the future. This patch would cause all >>>> of the remaining records to get thrown out, but the "future" record to be >>>> kept. >>>> >>>> In any case, it's way better than what we have, which allows negative >>>> intervals. >>>> >>>> -tk >>>> >>>> >>>> >>>> On Fri, Sep 14, 2018 at 3:53 PM Chris Manton <[email protected]> wrote: >>>> >>>>> I suppose if the archived packets are presented out of order there >>>>> could be a negative interval... >>>>> >>>>> I have not seen this but the wmr200 is full of wonderful surprises. >>>>> >>>>> One potential drive-by fix is to drop all packets satisfying this >>>>> failing condition...here is one untested fix: >>>>> >>>>> >>>>> $ git diff >>>>> diff --git a/bin/weewx/drivers/wmr200.py b/bin/weewx/drivers/wmr200.py >>>>> index eaab5a14..62b743e0 100644 >>>>> --- a/bin/weewx/drivers/wmr200.py >>>>> +++ b/bin/weewx/drivers/wmr200.py >>>>> @@ -1943,7 +1943,13 @@ class WMR200(weewx.drivers.AbstractDevice): >>>>> timestamp_packet_interval = timestamp_packet_current \ >>>>> - timestamp_packet_previous >>>>> >>>>> - if pkt.timestamp_record() > (timestamp_packet_previous >>>>> + if timestamp_packet_interval < 0: >>>>> + loginf(('genStartup() Discarding received archive' >>>>> + ' record that presented out-of-order' >>>>> + ' current_threshold:%s previous_timestamp:%s') >>>>> + % >>>>> (weeutil.weeutil.timestamp_to_string(pkt.timestamp_record()) >>>>> + >>>>> weeutil.weeutil.timestamp_to_string(timestamp_packet_previous))) >>>>> + elif pkt.timestamp_record() > >>>>> (timestamp_packet_previous >>>>> + >>>>> self._archive_threshold): >>>>> loginf(('genStartup() Discarding received archive' >>>>> ' record exceeding archive interval >>>>> cnt:%d' >>>>> >>>>> On Friday, September 14, 2018 at 6:07:48 AM UTC-7, Thomas Keffer wrote: >>>>>> >>>>>> Good. >>>>>> >>>>>> Looking through the WMR200 driver code (I didn't write it), it >>>>>> appears that a positive value for 'interval' is not guaranteed. I'll ask >>>>>> the author to take a look at it. >>>>>> >>>>>> -tk >>>>>> >>>>>> On Fri, Sep 14, 2018 at 5:44 AM Rahul Bedi <[email protected]> wrote: >>>>>> >>>>>>> Yup found a record with -ve interval, changed that to 5 as thats >>>>>>> what I am using. >>>>>>> >>>>>>> And problem fixed !! >>>>>>> >>>>>>> Thanks a lot. >>>>>>> >>>>>>> Not sure how this bad data entered into the db. >>>>>>> >>>>>>> Will be great if there is utility that finds such errors in the db >>>>>>> and flags them. >>>>>>> >>>>>>> - Rahul >>>>>>> >>>>>>> On Fri, Sep 14, 2018 at 5:48 PM Thomas Keffer <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> I think the thing to do is to figure out how a negative value got >>>>>>>> in there, fix that, then rebuild the daily summaries. >>>>>>>> >>>>>>>> One possibility is that a record in your database has a negative >>>>>>>> value for 'interval' --- the time length of an archive record. Let's >>>>>>>> check >>>>>>>> that. >>>>>>>> >>>>>>>> *sqlite3 /var/lib/weewx/weewx.sdb* >>>>>>>> sqlite> *SELECT datetime(dateTime, 'unixepoch','localtime'), >>>>>>>> interval FROM archive WHERE interval<0;* >>>>>>>> >>>>>>>> -tk >>>>>>>> >>>>>>>> On Fri, Sep 14, 2018 at 5:06 AM Rahul Bedi <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Here is what I get: >>>>>>>>> >>>>>>>>> 2018-08-02 00:00:00|2851473.0|86400 >>>>>>>>> 2018-08-03 00:00:00|2139447.0|86400 >>>>>>>>> 2018-08-04 00:00:00|2867211.0|86400 >>>>>>>>> 2018-08-05 00:00:00|3306195.0|86400 >>>>>>>>> 2018-08-06 00:00:00|3167394.0|86400 >>>>>>>>> 2018-08-07 00:00:00|2924283.0|86400 >>>>>>>>> 2018-08-08 00:00:00|2255208.0|86400 >>>>>>>>> 2018-08-09 00:00:00|2797065.0|86400 >>>>>>>>> 2018-08-10 00:00:00|2693187.0|86400 >>>>>>>>> 2018-08-11 00:00:00|2981601.0|86400 >>>>>>>>> 2018-08-12 00:00:00|3869136.0|86400 >>>>>>>>> 2018-08-13 00:00:00|3632787.0|86400 >>>>>>>>> 2018-08-14 00:00:00|2651829.0|86400 >>>>>>>>> 2018-08-15 00:00:00|2104410.0|86400 >>>>>>>>> 2018-08-16 00:00:00|3682674.0|86400 >>>>>>>>> 2018-08-17 00:00:00|3499992.0|86400 >>>>>>>>> 2018-08-18 00:00:00|3855531.0|86400 >>>>>>>>> 2018-08-19 00:00:00|3875121.0|86400 >>>>>>>>> 2018-08-20 00:00:00|3915861.0|86400 >>>>>>>>> 2018-08-21 00:00:00|3523719.0|86400 >>>>>>>>> 2018-08-22 00:00:00|1637909.95179334|85620 >>>>>>>>> 2018-08-23 00:00:00|506702.215048778|25380 >>>>>>>>> 2018-08-24 00:00:00|523285.18735593|101340 >>>>>>>>> 2018-08-25 00:00:00|2047231.90482286|92280 >>>>>>>>> 2018-08-26 00:00:00|2207680.59885907|85800 >>>>>>>>> 2018-08-27 00:00:00|2112653.43729687|78000 >>>>>>>>> 2018-08-28 00:00:00|1927695.38113441|85200 >>>>>>>>> 2018-08-29 00:00:00|1696867.80614767|76740 >>>>>>>>> 2018-08-30 00:00:00|1892572.6564943|85500 >>>>>>>>> 2018-08-31 00:00:00|1669207.56530998|86100 >>>>>>>>> 2018-09-01 00:00:00|1882876.90336366|85080 >>>>>>>>> 2018-09-02 00:00:00|1552040.97205686|86100 >>>>>>>>> 2018-09-03 00:00:00|1550916.37957731|86100 >>>>>>>>> 2018-09-04 00:00:00|1093704.34743737|85740 >>>>>>>>> 2018-09-05 00:00:00|1688380.4274756|86100 >>>>>>>>> 2018-09-06 00:00:00|2349485.94738889|86100 >>>>>>>>> 2018-09-07 00:00:00|2676307.36370007|85560 >>>>>>>>> 2018-09-08 00:00:00|2198034.86178985|86100 >>>>>>>>> 2018-09-09 00:00:00|-7107216.29856356|-302280 >>>>>>>>> 2018-09-10 00:00:00|288713.672694377|85560 >>>>>>>>> 2018-09-11 00:00:00|351094.880509644|85860 >>>>>>>>> 2018-09-12 00:00:00|441249.222795585|85500 >>>>>>>>> 2018-09-13 00:00:00|60033.1634844758|85620 >>>>>>>>> 2018-09-14 00:00:00|21222.2555021576|62820 >>>>>>>>> >>>>>>>>> I am guessing that the negative values above may be the culprit. >>>>>>>>> How are these populated in the archive_day_wind table or should I >>>>>>>>> just edit >>>>>>>>> these values out ? >>>>>>>>> >>>>>>>>> - Rahul >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Sep 14, 2018 at 5:19 PM Thomas Keffer <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> You're on the right track: there must be an bad value in your >>>>>>>>>> database. Did you, by any chance, use a visual sql editor? They >>>>>>>>>> often put >>>>>>>>>> null strings in the database. >>>>>>>>>> >>>>>>>>>> In any case, the field 'wsquaresum' and 'sumtime' are used by >>>>>>>>>> rms. So, try this: >>>>>>>>>> >>>>>>>>>> *sqlite3 /var/lib/weewx/weewx.sdb* >>>>>>>>>> sqlite> *SELECT datetime(dateTime, 'unixepoch','localtime'), >>>>>>>>>> wsquaresum, sumtime FROM archive_day_wind WHERE dateTime > >>>>>>>>>> 1533081600;* >>>>>>>>>> >>>>>>>>>> This will show all values from (roughly) 1-August (about 45 >>>>>>>>>> values). If the bad value is before that, we'll have to use a >>>>>>>>>> different >>>>>>>>>> strategy, rather than go looking for a needle in a haystack. >>>>>>>>>> >>>>>>>>>> -tk >>>>>>>>>> >>>>>>>>>> On Thu, Sep 13, 2018 at 11:35 PM Rahul Bedi <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I found rms used in the statictics.ico and hilo.ico files in the >>>>>>>>>>> Seasons directory. removed those segments. Diff as follows: >>>>>>>>>>> >>>>>>>>>>> $ diff statistics.14092018.inc statistics.inc >>>>>>>>>>> >>>>>>>>>>> 134,140d133 >>>>>>>>>>> < <tr> >>>>>>>>>>> < <td class="label">$obs.label.wind_rms</td> >>>>>>>>>>> < <td class="units">$unit.label.wind</td> >>>>>>>>>>> < #for $archive in $archive_data >>>>>>>>>>> < <td class="data >>>>>>>>>>> new_row">$archive.wind.rms.formatted</td> >>>>>>>>>>> < #end for >>>>>>>>>>> < </tr> >>>>>>>>>>> >>>>>>>>>>> $ diff hilo.inc hilo.14092018.inc >>>>>>>>>>> >>>>>>>>>>> 132a133,140 >>>>>>>>>>> > <tr> >>>>>>>>>>> > <td class="label">$obs.label.wind_rms</td> >>>>>>>>>>> > #for $archive in $archive_data >>>>>>>>>>> > <td class="data new_row hilo_$archive[0]"> >>>>>>>>>>> > $archive[1].wind.rms.formatted</td> >>>>>>>>>>> > #end for >>>>>>>>>>> > <td class="units">$unit.label.wind</td> >>>>>>>>>>> > </tr> >>>>>>>>>>> >>>>>>>>>>> Apart from this any file referring to rms were exactly the same >>>>>>>>>>> as the original install. Attaching skin.conf with this message. >>>>>>>>>>> >>>>>>>>>>> As expected the problem is gone but rms is not getting reported. >>>>>>>>>>> Which means the issue still stays. >>>>>>>>>>> >>>>>>>>>>> Could you help me with what fields in the archive are referred >>>>>>>>>>> to by rms ? I can try to manually check for any value discrepancy. >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> - Rahul >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, Sep 14, 2018 at 9:14 AM Rahul Bedi <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Thanks Gary. >>>>>>>>>>>> >>>>>>>>>>>> Charges were done about a month ago but have seen no errors >>>>>>>>>>>> till this episode. I will do a diff with the original skin files >>>>>>>>>>>> and post >>>>>>>>>>>> it here. >>>>>>>>>>>> >>>>>>>>>>>> Thanks. >>>>>>>>>>>> >>>>>>>>>>>> - Rahul >>>>>>>>>>>> >>>>>>>>>>>> On Fri 14 Sep, 2018, 7:35 AM gjr80, <[email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> On Friday, 14 September 2018 11:30:42 UTC+10, Rahul Bedi wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> As I have made no code or config changes since the day >>>>>>>>>>>>>> before, I believe it doing this for the right data. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> This sounds to me like you have changed some of the seasons >>>>>>>>>>>>> skin files. The error is coming form the use of a .rms tag in >>>>>>>>>>>>> one of the seasons .tmpl or .inc files. The seasons skin only >>>>>>>>>>>>> uses the .rms tag with the WeeWX wind field. As Tom indicated >>>>>>>>>>>>> .rms only works against the wind field, if your changes have >>>>>>>>>>>>> used the .rms tag against another field then this could be >>>>>>>>>>>>> the cause. I suggest you go through all of your changes to the >>>>>>>>>>>>> seasons skin >>>>>>>>>>>>> files and see what fields are being used with the .rms tag. >>>>>>>>>>>>> Until we know for sure, and given you seem to have made changes >>>>>>>>>>>>> to the >>>>>>>>>>>>> skin, we will likely be chasing our tails. >>>>>>>>>>>>> >>>>>>>>>>>>> Gary >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> You received this message because you are subscribed to a >>>>>>>>>>>>> topic in the Google Groups "weewx-user" group. >>>>>>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>>>>>> https://groups.google.com/d/topic/weewx-user/LWmSpcVfdmo/unsubscribe >>>>>>>>>>>>> . >>>>>>>>>>>>> To unsubscribe from this group and all its topics, send an >>>>>>>>>>>>> email to [email protected]. >>>>>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "weewx-user" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to a topic >>>>>>>>>> in the Google Groups "weewx-user" group. >>>>>>>>>> To unsubscribe from this topic, visit >>>>>>>>>> https://groups.google.com/d/topic/weewx-user/LWmSpcVfdmo/unsubscribe >>>>>>>>>> . >>>>>>>>>> To unsubscribe from this group and all its topics, send an email >>>>>>>>>> to [email protected]. >>>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "weewx-user" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to a topic in >>>>>>>> the Google Groups "weewx-user" group. >>>>>>>> To unsubscribe from this topic, visit >>>>>>>> https://groups.google.com/d/topic/weewx-user/LWmSpcVfdmo/unsubscribe >>>>>>>> . >>>>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>>>> [email protected]. >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "weewx-user" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "weewx-user" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/weewx-user/LWmSpcVfdmo/unsubscribe. >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to a topic in the > Google Groups "weewx-user" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/weewx-user/LWmSpcVfdmo/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
