I've ported wunderfixer to the new WU API. Commit 32c35ce <https://github.com/weewx/weewx/commit/32c35ce380b766054b05a7b64d1d82ab113a5d73> on the development branch.
Unfortunately, it looks like the WU no longer allows re-posting old records, so the utility may no longer be useful. I'd be interested in other people's experience. Also, as I understand it, there are issues for people living east of Greenwich. I live west, so I wasn't able to check that. Other people's experiences would be welcome. NB: this is on the *development* branch. You will have to clone the weewx GitHub repository, then check out the development branch. -tk On Monday, June 3, 2019 at 3:21:54 PM UTC-7, Rod Yager wrote: > > I’ve already migrated the today logic into my local version of > wunderfixer, which runs as a cron job at 05:58, 11:58, 17:58 and 23:58. > > This works as expected here. > > The only circumstance I can see in which it would break is if I make a > request for today’s data at 23:59:59.95 on my machine. The transmission > delay (and possibly differences in the clocks) will mean that WU receives > my request at a time when it thinks it is 00:00:00.01 on the following day > (my time) and so it will return the (empty) data for the day after the date > on which I made the request. We can’t worry about such things. > > > Rod > > > > On 4 Jun 2019, at 7:55 am, Leon Shaner <[email protected]> wrote: > > Thanks, Rod! =D > > Those are precisely the same tests I ran and exact same results that > noted, before before publishing. =D > Plus a ton of checks against my own station, KMIDEARB5, west of UTC. > > As before, even more important than these 'historical' API tests is to > query against the current day at different times throughout the day, such > as before and after your midnight and before and after UTC midnight. That > logic which uses the 'today' API (rapid) for today is the one we will > likely keep no matter what IBM does with the well-known bugs in the > 'historical' API. > > (I can't test the 'today' logic 100% without changing my localtime on my > system and I don't want to do that, which is why I appreciate help from > others to validate the code). =D > > Regards, > \Leon > -- > Leon Shaner :: Dearborn, Michigan (iPad Pro) > > On Jun 3, 2019, at 5:39 PM, Rod Yager <[email protected]> wrote: > > Dear Leon, > > Thanks for all your work on this. This works as well as we are going to be > able to manage unless and until WU fixes the bug. > > The remaining issue affects just one date, the “change-over” date when WU > transitions to returning the correct data - and will only bite for stations > east of UTC, and only in the hours that they are ahead of UTC. > > Currently, for me, that date is 2019-05-28. When it originally asks for > 2019-05-28, it returns the data for 2019-05-29. Wunderfixer then tries to > compensate by asking for 2019-05-27. But 2019-05-27 is a date before the WU > bug, and so it returns the data for 2019-05-27. We can’t work around this, > because no request to WU will actually return the data we want. The only > fix is to wait a few hours until the UTC date is aligned with the Sydney > date. > > Good news is that for stations west of UTC, this won’t happen as the bug > causes a duplicate day, not a missing day. > > Here’s the output for the requests for my station from 2019-05-27 (before > the WU bug kicks in) to today 2019-06-04. > > > [rodyager@moses ~]$ /home/weewx/bin/wunderdates3 --epsilon=125 --date > 2019-05-27 --test --verbose | (head -n 11 ; tail -n 3) > Using configuration file /home/weewx/weewx.conf. > Weather Underground Station: ISYDNEY155 > Date to check: 2019-05-27 > WU API obsTimeLocal: 2019-05-27 > epoch: 1558879200 date_epoch_local: 2019-05-27 00:00:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-26T14:00:00Z obsTimeLocal: 2019-05-27 > 00:00:00 > epoch: 1558879500 date_epoch_local: 2019-05-27 00:05:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-26T14:05:00Z obsTimeLocal: 2019-05-27 > 00:05:00 > epoch: 1558879800 date_epoch_local: 2019-05-27 00:10:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-26T14:10:00Z obsTimeLocal: 2019-05-27 > 00:10:00 > epoch: 1558880100 date_epoch_local: 2019-05-27 00:15:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-26T14:15:00Z obsTimeLocal: 2019-05-27 > 00:15:00 > epoch: 1558880400 date_epoch_local: 2019-05-27 00:20:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-26T14:20:00Z obsTimeLocal: 2019-05-27 > 00:20:00 > epoch: 1558880700 date_epoch_local: 2019-05-27 00:25:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-26T14:25:00Z obsTimeLocal: 2019-05-27 > 00:25:00 > epoch: 1558881000 date_epoch_local: 2019-05-27 00:30:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-26T14:30:00Z obsTimeLocal: 2019-05-27 > 00:30:00 > epoch: 1558965000 date_epoch_local: 2019-05-27 23:50:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-27T13:50:00Z obsTimeLocal: 2019-05-27 > 23:50:00 > epoch: 1558965300 date_epoch_local: 2019-05-27 23:55:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-27T13:55:00Z obsTimeLocal: 2019-05-27 > 23:55:00 > Number of WU records: 288 > [rodyager@moses ~]$ /home/weewx/bin/wunderdates3 --epsilon=125 --date > 2019-05-28 --test --verbose | (head -n 11 ; tail -n 3) > > No results returned from Weather Underground (perhaps a bad station > name??). > Using configuration file /home/weewx/weewx.conf. > Weather Underground Station: ISYDNEY155 > Date to check: 2019-05-28 > WU API obsTimeLocal: 2019-05-29 > WU API RETURNED WRONG DATE!!!!!!!!!!!!!!! > WU API COMPENSATION DATE: 2019-05-27 > WU API obsTimeLocal: 2019-05-27 > WU API COMPENSATION FAILURE! ABORTING!!! > Number of WU records: 0 > [rodyager@moses ~]$ /home/weewx/bin/wunderdates3 --epsilon=125 --date > 2019-05-29 --test --verbose | (head -n 11 ; tail -n 3) > Using configuration file /home/weewx/weewx.conf. > Weather Underground Station: ISYDNEY155 > Date to check: 2019-05-29 > WU API obsTimeLocal: 2019-05-30 > WU API RETURNED WRONG DATE!!!!!!!!!!!!!!! > WU API COMPENSATION DATE: 2019-05-28 > WU API obsTimeLocal: 2019-05-29 > epoch: 1559052000 date_epoch_local: 2019-05-29 00:00:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-28T14:00:00Z obsTimeLocal: 2019-05-29 > 00:00:00 > epoch: 1559052300 date_epoch_local: 2019-05-29 00:05:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-28T14:05:00Z obsTimeLocal: 2019-05-29 > 00:05:00 > epoch: 1559052600 date_epoch_local: 2019-05-29 00:10:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-28T14:10:00Z obsTimeLocal: 2019-05-29 > 00:10:00 > epoch: 1559052900 date_epoch_local: 2019-05-29 00:15:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-28T14:15:00Z obsTimeLocal: 2019-05-29 > 00:15:00 > epoch: 1559137800 date_epoch_local: 2019-05-29 23:50:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-29T13:50:00Z obsTimeLocal: 2019-05-29 > 23:50:00 > epoch: 1559138100 date_epoch_local: 2019-05-29 23:55:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-29T13:55:00Z obsTimeLocal: 2019-05-29 > 23:55:00 > Number of WU records: 288 > [rodyager@moses ~]$ /home/weewx/bin/wunderdates3 --epsilon=125 --date > 2019-05-30 --test --verbose | (head -n 11 ; tail -n 3) > Using configuration file /home/weewx/weewx.conf. > Weather Underground Station: ISYDNEY155 > Date to check: 2019-05-30 > WU API obsTimeLocal: 2019-05-31 > WU API RETURNED WRONG DATE!!!!!!!!!!!!!!! > WU API COMPENSATION DATE: 2019-05-29 > WU API obsTimeLocal: 2019-05-30 > epoch: 1559138400 date_epoch_local: 2019-05-30 00:00:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-29T14:00:00Z obsTimeLocal: 2019-05-30 > 00:00:00 > epoch: 1559138700 date_epoch_local: 2019-05-30 00:05:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-29T14:05:00Z obsTimeLocal: 2019-05-30 > 00:05:00 > epoch: 1559139000 date_epoch_local: 2019-05-30 00:10:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-29T14:10:00Z obsTimeLocal: 2019-05-30 > 00:10:00 > epoch: 1559139300 date_epoch_local: 2019-05-30 00:15:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-29T14:15:00Z obsTimeLocal: 2019-05-30 > 00:15:00 > epoch: 1559224200 date_epoch_local: 2019-05-30 23:50:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-30T13:50:00Z obsTimeLocal: 2019-05-30 > 23:50:00 > epoch: 1559224500 date_epoch_local: 2019-05-30 23:55:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-30T13:55:00Z obsTimeLocal: 2019-05-30 > 23:55:00 > Number of WU records: 288 > [rodyager@moses ~]$ /home/weewx/bin/wunderdates3 --epsilon=125 --date > 2019-05-31 --test --verbose | (head -n 11 ; tail -n 3) > Using configuration file /home/weewx/weewx.conf. > Weather Underground Station: ISYDNEY155 > Date to check: 2019-05-31 > WU API obsTimeLocal: 2019-06-01 > WU API RETURNED WRONG DATE!!!!!!!!!!!!!!! > WU API COMPENSATION DATE: 2019-05-30 > WU API obsTimeLocal: 2019-05-31 > epoch: 1559224800 date_epoch_local: 2019-05-31 00:00:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-30T14:00:00Z obsTimeLocal: 2019-05-31 > 00:00:00 > epoch: 1559225100 date_epoch_local: 2019-05-31 00:05:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-30T14:05:00Z obsTimeLocal: 2019-05-31 > 00:05:00 > epoch: 1559225400 date_epoch_local: 2019-05-31 00:10:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-30T14:10:00Z obsTimeLocal: 2019-05-31 > 00:10:00 > epoch: 1559225700 date_epoch_local: 2019-05-31 00:15:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-30T14:15:00Z obsTimeLocal: 2019-05-31 > 00:15:00 > epoch: 1559310600 date_epoch_local: 2019-05-31 23:50:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-31T13:50:00Z obsTimeLocal: 2019-05-31 > 23:50:00 > epoch: 1559310900 date_epoch_local: 2019-05-31 23:55:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-31T13:55:00Z obsTimeLocal: 2019-05-31 > 23:55:00 > Number of WU records: 288 > [rodyager@moses ~]$ /home/weewx/bin/wunderdates3 --epsilon=125 --date > 2019-06-01 --test --verbose | (head -n 11 ; tail -n 3) > Using configuration file /home/weewx/weewx.conf. > Weather Underground Station: ISYDNEY155 > Date to check: 2019-06-01 > WU API obsTimeLocal: 2019-06-02 > WU API RETURNED WRONG DATE!!!!!!!!!!!!!!! > WU API COMPENSATION DATE: 2019-05-31 > WU API obsTimeLocal: 2019-06-01 > epoch: 1559311200 date_epoch_local: 2019-06-01 00:00:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-31T14:00:00Z obsTimeLocal: 2019-06-01 > 00:00:00 > epoch: 1559311500 date_epoch_local: 2019-06-01 00:05:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-31T14:05:00Z obsTimeLocal: 2019-06-01 > 00:05:00 > epoch: 1559311800 date_epoch_local: 2019-06-01 00:10:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-31T14:10:00Z obsTimeLocal: 2019-06-01 > 00:10:00 > epoch: 1559312100 date_epoch_local: 2019-06-01 00:15:00 tz: > Australia/Sydney obsTimeUtc: 2019-05-31T14:15:00Z obsTimeLocal: 2019-06-01 > 00:15:00 > epoch: 1559397000 date_epoch_local: 2019-06-01 23:50:00 tz: > Australia/Sydney obsTimeUtc: 2019-06-01T13:50:00Z obsTimeLocal: 2019-06-01 > 23:50:00 > epoch: 1559397300 date_epoch_local: 2019-06-01 23:55:00 tz: > Australia/Sydney obsTimeUtc: 2019-06-01T13:55:00Z obsTimeLocal: 2019-06-01 > 23:55:00 > Number of WU records: 288 > [rodyager@moses ~]$ /home/weewx/bin/wunderdates3 --epsilon=125 --date > 2019-06-02 --test --verbose | (head -n 11 ; tail -n 3) > Using configuration file /home/weewx/weewx.conf. > Weather Underground Station: ISYDNEY155 > Date to check: 2019-06-02 > WU API obsTimeLocal: 2019-06-03 > WU API RETURNED WRONG DATE!!!!!!!!!!!!!!! > WU API COMPENSATION DATE: 2019-06-01 > WU API obsTimeLocal: 2019-06-02 > epoch: 1559397600 date_epoch_local: 2019-06-02 00:00:00 tz: > Australia/Sydney obsTimeUtc: 2019-06-01T14:00:00Z obsTimeLocal: 2019-06-02 > 00:00:00 > epoch: 1559397900 date_epoch_local: 2019-06-02 00:05:00 tz: > Australia/Sydney obsTimeUtc: 2019-06-01T14:05:00Z obsTimeLocal: 2019-06-02 > 00:05:00 > epoch: 1559398200 date_epoch_local: 2019-06-02 00:10:00 tz: > Australia/Sydney obsTimeUtc: 2019-06-01T14:10:00Z obsTimeLocal: 2019-06-02 > 00:10:00 > epoch: 1559398500 date_epoch_local: 2019-06-02 00:15:00 tz: > Australia/Sydney obsTimeUtc: 2019-06-01T14:15:00Z obsTimeLocal: 2019-06-02 > 00:15:00 > epoch: 1559483400 date_epoch_local: 2019-06-02 23:50:00 tz: > Australia/Sydney obsTimeUtc: 2019-06-02T13:50:00Z obsTimeLocal: 2019-06-02 > 23:50:00 > epoch: 1559483700 date_epoch_local: 2019-06-02 23:55:00 tz: > Australia/Sydney obsTimeUtc: 2019-06-02T13:55:00Z obsTimeLocal: 2019-06-02 > 23:55:00 > Number of WU records: 288 > [rodyager@moses ~]$ /home/weewx/bin/wunderdates3 --epsilon=125 --date > 2019-06-03 --test --verbose | (head -n 11 ; tail -n 3) > Using configuration file /home/weewx/weewx.conf. > Weather Underground Station: ISYDNEY155 > Date to check: 2019-06-03 > WU API obsTimeLocal: 2019-06-04 > WU API RETURNED WRONG DATE!!!!!!!!!!!!!!! > WU API COMPENSATION DATE: 2019-06-02 > WU API obsTimeLocal: 2019-06-03 > epoch: 1559484000 date_epoch_local: 2019-06-03 00:00:00 tz: > Australia/Sydney obsTimeUtc: 2019-06-02T14:00:00Z obsTimeLocal: 2019-06-03 > 00:00:00 > epoch: 1559484300 date_epoch_local: 2019-06-03 00:05:00 tz: > Australia/Sydney obsTimeUtc: 2019-06-02T14:05:00Z obsTimeLocal: 2019-06-03 > 00:05:00 > epoch: 1559484600 date_epoch_local: 2019-06-03 00:10:00 tz: > Australia/Sydney obsTimeUtc: 2019-06-02T14:10:00Z obsTimeLocal: 2019-06-03 > 00:10:00 > epoch: 1559484900 date_epoch_local: 2019-06-03 00:15:00 tz: > Australia/Sydney obsTimeUtc: 2019-06-02T14:15:00Z obsTimeLocal: 2019-06-03 > 00:15:00 > epoch: 1559569800 date_epoch_local: 2019-06-03 23:50:00 tz: > Australia/Sydney obsTimeUtc: 2019-06-03T13:50:00Z obsTimeLocal: 2019-06-03 > 23:50:00 > epoch: 1559570100 date_epoch_local: 2019-06-03 23:55:00 tz: > Australia/Sydney obsTimeUtc: 2019-06-03T13:55:00Z obsTimeLocal: 2019-06-03 > 23:55:00 > Number of WU records: 288 > [rodyager@moses ~]$ /home/weewx/bin/wunderdates3 --epsilon=125 --test > --verbose | (head -n 11 ; tail -n 3) > Using configuration file /home/weewx/weewx.conf. > Weather Underground Station: ISY > > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/641183af-69e7-4c99-bc7d-f8092554194d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
