So this is where I am... These all seem reasonable (but then scroll for the problem which starts with yesterday's date):
pi@nixie:/usr/share/weewx4/bin $ ./wunderfixer --epsilon=125 --date=2019-05-21 --test --verbose | more Using configuration file /usr/share/weewx4/weewx.conf. Using database binding 'wx_binding', which is bound to database 'archive_sqlite' Weather Underground Station: KMIDEARB5 Date to check: 2019-05-21 Number of archive records: 1440 Number of WU records: 288 Number of missing records: 4 Missing records: 2019-05-21 00:00:00 EDT (1558411200); 29.379"; 49.8F; 78%; 0.0 mph; N/A deg; 0.0 mph gust; 43.2F; 0.00" rain ... skipped. 2019-05-21 00:01:00 EDT (1558411260); 29.379"; 49.7F; 78%; 0.0 mph; N/A deg; 0.0 mph gust; 43.2F; 0.00" rain ... skipped. 2019-05-21 00:02:00 EDT (1558411320); 29.379"; 49.6F; 78%; 0.0 mph; N/A deg; 0.0 mph gust; 43.1F; 0.00" rain ... skipped. 2019-05-21 22:42:00 EDT (1558492920); 29.406"; 55.4F; 67%; 1.3 mph; 68 deg; 0.0 mph gust; 44.6F; 0.00" rain ... skipped. pi@nixie:/usr/share/weewx4/bin $ ./wunderfixer --epsilon=125 --date=2019-05-22 --test --verbose | more Using configuration file /usr/share/weewx4/weewx.conf. Using database binding 'wx_binding', which is bound to database 'archive_sqlite' Weather Underground Station: KMIDEARB5 Date to check: 2019-05-22 Number of archive records: 1440 Number of WU records: 288 Number of missing records: 3 Missing records: 2019-05-22 00:00:00 EDT (1558497600); 29.406"; 54.0F; 65%; 2.0 mph; 76 deg; 3.6 mph gust; 42.5F; 0.00" rain ... skipped. 2019-05-22 00:01:00 EDT (1558497660); 29.406"; 54.0F; 65%; 2.3 mph; 64 deg; 2.5 mph gust; 42.5F; 0.00" rain ... skipped. 2019-05-22 00:02:00 EDT (1558497720); 29.406"; 54.0F; 65%; 2.5 mph; 61 deg; 2.9 mph gust; 42.5F; 0.00" rain ... skipped. pi@nixie:/usr/share/weewx4/bin $ ./wunderfixer --epsilon=125 --date=2019-05-23 --test --verbose | more Using configuration file /usr/share/weewx4/weewx.conf. Using database binding 'wx_binding', which is bound to database 'archive_sqlite' Weather Underground Station: KMIDEARB5 Date to check: 2019-05-23 Number of archive records: 1438 Number of WU records: 288 Number of missing records: 4 Missing records: 2019-05-23 00:00:00 EDT (1558584000); 29.244"; 59.9F; 90%; 1.3 mph; 158 deg; 1.8 mph gust; 56.9F; 0.00" rain ... skipped. 2019-05-23 00:01:00 EDT (1558584060); 29.244"; 59.8F; 90%; 1.3 mph; 159 deg; 1.8 mph gust; 56.8F; 0.00" rain ... skipped. 2019-05-23 00:02:00 EDT (1558584120); 29.244"; 59.7F; 90%; 1.3 mph; 160 deg; 1.3 mph gust; 56.8F; 0.00" rain ... skipped. 2019-05-23 14:52:00 EDT (1558637520); 29.161"; 83.8F; 46%; 3.6 mph; 248 deg;12.3 mph gust; 60.8F; 0.00" rain ... skipped. Using configuration file /usr/share/weewx4/weewx.conf. Using database binding 'wx_binding', which is bound to database 'archive_sqlite' Weather Underground Station: KMIDEARB5 Date to check: 2019-05-24 Number of archive records: 1436 Number of WU records: 260 Number of missing records: 149 Missing records: 2019-05-24 00:00:00 EDT (1558670400); 29.320"; 63.0F; 70%; 0.0 mph; N/A deg; 0.0 mph gust; 52.9F; 0.00" rain ... skipped. 2019-05-24 00:01:00 EDT (1558670460); 29.320"; 63.0F; 70%; 0.0 mph; N/A deg; 0.0 mph gust; 52.9F; 0.00" rain ... skipped. 2019-05-24 00:02:00 EDT (1558670520); 29.320"; 63.0F; 70%; 0.0 mph; N/A deg; 0.0 mph gust; 52.9F; 0.00" rain ... skipped. 2019-05-24 18:22:00 EDT (1558736520); 29.359"; 67.5F; 68%; 2.5 mph; 99 deg; 5.1 mph gust; 56.5F; 0.00" rain ... skipped. 2019-05-24 18:40:00 EDT (1558737600); 29.353"; 67.3F; 68%; 2.0 mph; 118 deg; 4.0 mph gust; 56.3F; 0.00" rain ... skipped. 2019-05-24 18:41:00 EDT (1558737660); 29.353"; 67.3F; 68%; 2.0 mph; 118 deg; 3.6 mph gust; 56.3F; 0.00" rain ... skipped. ... When forming the query URL, I suspect I need to convert the local date to the UTC date, since I was within 4 hours of UTC midnight. The docs will probably confirm it. The thing is, late last night (US/Eastern), the number was over 1300. If it's the UTC vs. EDT date in the URL, I would expect the records to line up when querying what is querying what is now yesterday. And for today, so far, seems reasonable. pi@nixie:/usr/share/weewx4/bin $ ./wunderfixer --epsilon=125 --date=2019-05-25 --test --verbose | more Using configuration file /usr/share/weewx4/weewx.conf. Using database binding 'wx_binding', which is bound to database 'archive_sqlite' Weather Underground Station: KMIDEARB5 Date to check: 2019-05-25 Number of archive records: 526 Number of WU records: 106 Number of missing records: 3 Missing records: 2019-05-25 00:00:00 EDT (1558756800); 29.329"; 61.4F; 75%; 0.0 mph; N/A deg; 0.0 mph gust; 53.4F; 0.00" rain ... skipped. 2019-05-25 00:01:00 EDT (1558756860); 29.329"; 61.3F; 75%; 0.0 mph; N/A deg; 3.1 mph gust; 53.3F; 0.00" rain ... skipped. 2019-05-25 00:02:00 EDT (1558756920); 29.329"; 61.3F; 75%; 0.0 mph; N/A deg; 2.5 mph gust; 53.3F; 0.00" rain ... skipped. Once I figure out what really happened with yesterday's data, late last night, and If I manage to find some long-lost brain cells over the weekend, I'd still like to "normalize" the local records to 5-minute boundaries, and not bother to flag the in-between ones for re-upload, since WU seems to drop them on purpose. There is no point re-uploading 00:00:00, then 00:01:00, then 00:02:00, because WU is only going to keep 00:00:00, and then later it will keep whatever is closest to 00:05:00. That's been the behavior of wunderfixer vs. WU for as long as I've been working with it, however. :-/ Regards, \Leon -- Leon Shaner :: Dearborn, Michigan (iPad Pro) > On May 25, 2019, at 8:08 AM, Leon Shaner <l...@isylum.org> wrote: > > Hey, Gary, > > I figured this out last night with a cue from Tom. But it requires modifying > two different classes in restx.py, similar to the way you mentioned and I am > a little uncomfortable with it, since I don't know all the different touch > points / ramifications for other code. > > Also, something strange started happening toward the end last night and the > list of missing records grew to pretty much 100%. > I double checked and the new query is still pulling back the expected ~288 > records (every 5 mins, is 12 an hour, so 288 in a day), but I now need to > scrutinize the "epoch" values, because it's almost as if IBM changed them by > some offset in the middle of the day, so now none of them "line up" with my > local data. > > I am heading out of town, so I will have to pick this up on Monday. > > Thanks for the help! =D > > Regards, > \Leon > -- > Leon Shaner :: Dearborn, Michigan (iPhone) > >> On May 25, 2019, at 1:24 AM, gjr80 <gjroder...@gmail.com> wrote: >> >> No, all get_site_dict does is check that the listed config options exist and >> are not the install defaults. The real issue is that _ambient_dict is used >> to pass keyword arguments to class AmbientThread init, apiKey is not a >> parameter in the init signature so hence the error. One approach will be to >> simply add an apiKey=None parameter to the AmbientClass init signature, it >> won't be used but who knows where WU may end up, maybe it could be thought >> of as a bit of future proofing/planning. Another approach would be to have a >> separate [Wunderfixer] stanza in weewx.conf, seems a waste to me and it >> makes sense to have the WU parameters together in the one place. Maybe leave >> it as a command line parameter only for wunderfixer, to me that seems almost >> the same as having a separate [Wunderfixer] stanza. Maybe introduce whatever >> approach in 4.0 and leave it as a wunderfixer command line parameter for the >> (expected) 3.9.2 release. Probably one for guidance from Tom. >> >> Gary >> >> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to weewx-user+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/weewx-user/68bfe21d-d0a2-4c55-9e71-2d65a69eea03%40googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to weewx-user+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/CAA23789-2FE9-4DCA-A2EC-386451439DC4%40isylum.org. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/4F4B3374-A489-4DF2-8E7B-3E2ABD71B676%40isylum.org. For more options, visit https://groups.google.com/d/optout.
