Rod, It's interesting that the "workaround" that I've already done, is actually working 100% for me. I got good results when combining the 'today' API when asking for the current date, and only using the 'historical' result when asking for dates other than the current date -- relative to station local time.
Here is my KMIDEARB5 station's transition from 7:59 p.m. to 8:00 p.m, where time moves to within the UTC-4 offset: $ diff 59:19:1:6_wu.txt 0:20:1:6_wu.txt 241c241,242 < Number of WU records: 239 --- > epoch: 1559433599 date_epoch_local: 2019-06-01 19:59:59 utcepoch: 1559433999 > date_epoch_utc: 2019-06-01 23:59:59 tz: America/New_York obsTimeUtc: > 2019-06-01T23:59:59Z obsTimeLocal: 2019-06-01 19:59:59 > Number of WU records: 240 The above is exactly correct -- the second query is one minute later and is after/on the 5-minute record "boundary" hence one additional record. And then as my station transition from 11:59 p.m. to midnight next day. Before midnight the 'today' API is used, but after midnight the 'historical' API will be used, which was always working for me after midnight (station local time): $ tail -n 4 59:23:1:6_wu.txt 0:0:2:6_wu.txt ==> 59:23:1:6_wu.txt <== epoch: 1559447399 date_epoch_local: 2019-06-01 23:49:59 utcepoch: 1559447799 date_epoch_utc: 2019-06-02 03:49:59 tz: America/New_York obsTimeUtc: 2019-06-02T03:49:59Z obsTimeLocal: 2019-06-01 23:49:59 epoch: 1559447699 date_epoch_local: 2019-06-01 23:54:59 utcepoch: 1559448099 date_epoch_utc: 2019-06-02 03:54:59 tz: America/New_York obsTimeUtc: 2019-06-02T03:54:59Z obsTimeLocal: 2019-06-01 23:54:59 epoch: 1559447701 date_epoch_local: 2019-06-01 23:55:01 utcepoch: 1559448101 date_epoch_utc: 2019-06-02 03:55:01 tz: America/New_York obsTimeUtc: 2019-06-02T03:55:01Z obsTimeLocal: 2019-06-01 23:55:01 Number of WU records: 288 ==> 0:0:2:6_wu.txt <== epoch: 1559447399 date_epoch_local: 2019-06-01 23:49:59 utcepoch: 1559447799 date_epoch_utc: 2019-06-02 03:49:59 tz: America/New_York obsTimeUtc: 2019-06-02T03:49:59Z obsTimeLocal: 2019-06-01 23:49:59 epoch: 1559447699 date_epoch_local: 2019-06-01 23:54:59 utcepoch: 1559448099 date_epoch_utc: 2019-06-02 03:54:59 tz: America/New_York obsTimeUtc: 2019-06-02T03:54:59Z obsTimeLocal: 2019-06-01 23:54:59 epoch: 1559447999 date_epoch_local: 2019-06-01 23:59:59 utcepoch: 1559448399 date_epoch_utc: 2019-06-02 03:59:59 tz: America/New_York obsTimeUtc: 2019-06-02T03:59:59Z obsTimeLocal: 2019-06-01 23:59:59 Number of WU records: 288 There again, the second query is a minute later, so one more result, but with explanation... ;-) There is a slight difference between the 'today' and 'historical' result. The first result, using the 'today' API ran at 1 minute before midnight, which was on the cusp of posting the 1-minute record nearest actual midnight. The 'today' result shows two time-stamps very close to 11:55 p.m. Where-as the 'historical' result replaces the later ~11:55 p.m. result with the ~midnight result. Those differences are "in the noise" and inconsequential. Even the default wunderfixer --epsilon=120 will filter out such a duplicate. I am happy that this new approach with the combined 'today' and 'historical' queries is at least effective for stations west of the UTC dateline. It means I have easily worked around half of the WU API bug. Now to further press IBM to fix the other half of the bug... =D (Even while I continue to think about ways to *cleanly* workaround the bug for stations east of UTC dateline). Regards, \Leon -- Leon Shaner :: Dearborn, Michigan (iPad Pro) > On Jun 1, 2019, at 11:08 PM, Rod Yager <[email protected]> wrote: > > Dear Leon, > > If WU doesn’t fix their bug, the solution is really quite simple. > > We need to make 3 requests to WU for the data: one request for the date we > are interested in, one request for the day before and one request for the day > after. > > eg. If we are intending to upload missing data for 2019-01-01 we need to ask > WU for the records it has for 2018-12-31, 2019-01-01 and 2019-01-02. > > One of these will have the information pertaining to the day we are > interested in. > > We then need to filter the three responses from WU, only keeping the > responses that pertain to the day we are interested in. > > At this point, we have exactly what we want from WU and so we compare that > with the local database, and upload the data that is missing in the WU > record, just as we always did. > > > in your timezone (UTC-4) if you ask for data for your station the correct > information will be in > the same day request: if the request is made between 00:00 and 19:59 (at > those times, the UTC date is the same as the date where you are ) > the request for the next day: if the request is made between 20:00 and 23:59 > (at those times, the UTC date is one day ahead the date where you are) > > If you request information for my station (I’ve sent you the details in a > PM), the correct information will be in > the same day request: if the request is made between 10:00 and 19:59 your > time (at those times, the UTC date is the same as the date where my station > is located) > the request for the previous day: if the request is made between 0:00 and > 09:59 or between 20:00 and 23:59 your time (at those times, the UTC date is > one day behind the date where my station is located) > > Rod > > > >> On 2 Jun 2019, at 9:27 am, Leon Shaner <[email protected]> wrote: >> >> Rod, >> >> Thanks for testing. Well, WU's approach is flawed, not mine, really. :S >> >> It means that both API's, the one I just added for 'today" and the one from >> before that was for 'historical' (including today), both have the same bug. >> >> For now, the best workaround I can think of is to only ever run the new >> wunderfixer only ever for the previous day, only ever after UTC has rolled >> over midnight. :-( >> E.g. when it is 2019-06-02 in UTC, you can get the right results with >> wunderfixer if you ask for 2019-06-01 only. In other words, always delay >> your wunderfixer fix ups by a day, operating always on yesterday's date >> (relative to UTC) after UTC has rolled over to the next day. >> >> Scroll to see the output of me running wunderdates on my local machine in >> UTC-4 against your station in UTC+10. >> >> Because I am asking for 2019-06-01, which is also my local date, the logic >> will use the URL that queries for 'today' and instead of returning >> obsTimeLocal records, it is returning obsTimeUtc records for the requested >> date. We can see that it's returning 2019-06-02 relative to your station, >> even though I asked for 2019-06-01. >> >> Sure, I could go a bit further and use a related API query to check the >> timezone of your station and use date conversions to check if the date >> requested is 'today' relative to the station requested, but really this is >> intended to be run on the same machine as weewx, so the localtime date >> should match that of the station. It means there isn't any real reason for >> me to workaround their bug. In fact even if I did, then the logic would >> just fall through to the historical data URL, which we already know is buggy. >> >> For now, I'll just report this to IBM a bug in the 'today' API, similar to >> the bug in the 'historical' API. >> >> $ ./wunderdates --epsilon=125 --date 2019-06-01 --station ISYDNEY155 --test >> --verbose | more >> >> Using configuration file /usr/share/weewx4/weewx.conf. >> Weather Underground Station: ISYDNEY155 >> Date to check: 2019-06-01 >> epoch: 1559397600 date_epoch_local: 2019-06-01 10:00:00 utcepoch: 1559398000 >> date_epoch_utc: 2019-06-01 14:00:00 tz: Australia/Sydney obsTimeUtc: >> 2019-06-01T14:00:00Z obsTimeLocal: 2019-06-02 00:00:00 >> epoch: 1559397900 date_epoch_local: 2019-06-01 10:05:00 utcepoch: 1559398300 >> date_epoch_utc: 2019-06-01 14:05:00 tz: Australia/Sydney obsTimeUtc: >> 2019-06-01T14:05:00Z obsTimeLocal: 2019-06-02 00:05:00 >> ... >> >> >> Regards, >> \Leon >> -- >> Leon Shaner :: Dearborn, Michigan (iPad Pro) >> >> On Jun 1, 2019, at 6:03 PM, Rod Yager <[email protected]> wrote: >> >>> Dear Leon, >>> >>> Your approach is flawed. It won’t work for historical data. >>> >>> If you run >>> >>> wunderdates —date=2019-01-01 at 9pm your time you will receive the data >>> for 2018-12-31 (because your station is a day behind UTC at the time of >>> the request, so it gives you the data for the previous day). >>> >>> On the other hand, if you run >>> >>> wunderdates —date=2019-01-01 at 9am your time you will receive the data >>> for 2019-01-01 (because your station is in the same day as UTC at the time >>> of the request). >>> >>> If I do the same thing (I’m in UTC+10) >>> >>> wunderdates —date=2019-01-01 at 9pm my time produces the data for >>> 2019-01-01 (because my station is in the same day as UTC at the time of the >>> request) but >>> wunderdates —date=2019-01-01 at 9am my time produces the data for >>> 2019-01-02 (because my station is a day ahead of UTC at the time of the >>> request) >>> >>> Rod >>> >>>> On 2 Jun 2019, at 3:17 am, Leon Shaner <[email protected]> wrote: >>>> >>>> All, >>>> >>>> I'm testing a new approach. Below you will find links to an updated >>>> wunderdates utility that can be used to validate whether I am on the right >>>> track. >>>> The wunderdates utility simply dumps out timestamp related records from >>>> what WU returns from the query. >>>> >>>> We're looking mainly at the obsTimeUtc and obsTimeLocal values, which were >>>> demonstrated under the prior approach to be returning the wrong dates when >>>> within the stations localtime +/- offset from UTC. >>>> >>>> The new approach is to detect if the requested date is 'today' and if so, >>>> use a different API URL that already assumes 'today' and will hopefully >>>> not be subject to the UTC offset bugs we've been chasing with the >>>> historical data API URL. >>>> >>>> I have my crontab set to do another test approaching my UTC offset, just >>>> after coming within the offset, and then again just before and after >>>> midnight localtime. >>>> (Same test I did before, but now with the new approach in place). >>>> >>>> Here is the utility for anyone else that wants to check out the behavior: >>>> >>>> https://github.com/UberEclectic/weewx/blob/wuapi/bin/wunderdates3 >>>> https://github.com/UberEclectic/weewx/blob/wuapi/bin/wunderdates4 >>>> >>>> Which version you pull will depend on which base weewx you are running. >>>> Pull the one that matches your weewx version and place it in bin, next to >>>> wunderfixer, and it will take the same arguments as wunderfixer. >>>> >>>> You can just try wunderfixer as you normally would (with --apikey) and >>>> then run wunderdates(3 or 4) with exact same arguments to be able to see >>>> what actual timestamps WU is sending back for the date queried. >>>> Parameters like --epsilon don't have any effect in the case of >>>> wunderdates, but I left it there so you don't have to change options when >>>> running the util to get the debug output. >>>> >>>> Regards, >>>> \Leon >>>> -- >>>> Leon Shaner :: Dearborn, Michigan (iPad Pro) >>>> >>>> On May 30, 2019, at 12:40 AM, Leon Shaner <[email protected]> wrote: >>>> >>>>> Rod, >>>>> Thanks again for this. >>>>> >>>>> Since the in-progress version of wunderfixer doesn't really show you the >>>>> dates that come back from WU, I have written a tool just to debug the >>>>> dates. >>>>> >>>>> The command-line input and basing of defaults on weewx.conf works the >>>>> same as wunderfixer, except it doesn't look at your DB at all. It only >>>>> prints out datestamps in various incarnations coming back from WU and as >>>>> compared to your system's localtime. >>>>> >>>>> It would be helpful to see the "wunderdates" output at times like you've >>>>> shown below, a la before and after your localtime rolls around to UTC >>>>> midnight. >>>>> >>>>> Since you are at UTC + 10, another interesting time would be 11+ hours on >>>>> either side of UTC midnight, in addition to within 9 or less hours. This >>>>> is just to make sure we're covering all the corner cases. >>>>> >>>>> Gary reported a difference for stations that are east vs. west of GMT, >>>>> and I expect we're really chasing the same bug in that there is some bad >>>>> math WU is doing based on UTC offset, but since an offset can be +/-, the >>>>> effects go in opposite directions date-wise, depending on which side of >>>>> the UTC dateline your station is located. >>>>> >>>>> At least that is what I surmise may be happening, but the wunderdates >>>>> utility should shed light one way or the other. =D >>>>> >>>>> The wunderdates utility is available at the links below. >>>>> >>>>> https://github.com/UberEclectic/weewx/blob/wuapi/bin/wunderdates3 >>>>> https://github.com/UberEclectic/weewx/blob/wuapi/bin/wunderdates4 >>>>> >>>>> Which version you pull will depend on which base weewx you are running. >>>>> Pull the one that matches your weewx version and place it in bin, next to >>>>> wunderfixer, and it will take the same arguments as wunderfixer. >>>>> >>>>> You can just try wunderfixer as you normally would (with --apikey) and >>>>> then run wunderdates(3 or 4) with exact same arguments to be able to see >>>>> what actual timestamps WU is sending back for the date queried. >>>>> Parameters like --epsilon don't have any effect in the case of >>>>> wunderdates, but I left it there so you don't have to change options when >>>>> running the util to get the debug output. >>>>> >>>>> What I've been doing is saving the output to files for use with sdiff >>>>> (side-by-side) diff. >>>>> Or you can just compare the head and tail of each file individually. >>>>> >>>>> Example for my system before and after my local time on 5-28 once it was >>>>> at/after 8 p.m. here, which is within 4 hours of UTC midnight (I am at >>>>> UTC-4). >>>>> >>>>> This output is optimized for screen widths 203 or wider. Sorry. :S >>>>> Mainly the last two data fields in the output tell what we need to know. >>>>> >>>>> >>>>> pi@nixie:/var/tmp $ head -n 3 59:19:28:5_wu.txt 0:20:28:5_wu.txt >>>>> ==> 59:19:28:5_wu.txt <== >>>>> Using configuration file /usr/share/weewx4/weewx.conf. >>>>> epoch: 1559016299 date_epoch_local: 2019-05-28 00:04:59 utcepoch: >>>>> 1559016699 date_epoch_utc: 2019-05-28 04:04:59 tz: America/New_York >>>>> obsTimeUtc: 2019-05-28T04:04:59Z obsTimeLocal: 2019-05-28 00:04:59 >>>>> epoch: 1559016599 date_epoch_local: 2019-05-28 00:09:59 utcepoch: >>>>> 1559016999 date_epoch_utc: 2019-05-28 04:09:59 tz: America/New_York >>>>> obsTimeUtc: 2019-05-28T04:09:59Z obsTimeLocal: 2019-05-28 00:09:59 >>>>> >>>>> ==> 0:20:28:5_wu.txt <== >>>>> Using configuration file /usr/share/weewx4/weewx.conf. >>>>> epoch: 1558929899 date_epoch_local: 2019-05-27 00:04:59 utcepoch: >>>>> 1558930299 date_epoch_utc: 2019-05-27 04:04:59 tz: America/New_York >>>>> obsTimeUtc: 2019-05-27T04:04:59Z obsTimeLocal: 2019-05-27 00:04:59 >>>>> epoch: 1558930199 date_epoch_local: 2019-05-27 00:09:59 utcepoch: >>>>> 1558930599 date_epoch_utc: 2019-05-27 04:09:59 tz: America/New_York >>>>> obsTimeUtc: 2019-05-27T04:09:59Z obsTimeLocal: 2019-05-27 00:09:59 >>>>> >>>>> >>>>> pi@nixie:/var/tmp $ tail -n 3 59:19:28:5_wu.txt 0:20:28:5_wu.txt >>>>> ==> 59:19:28:5_wu.txt <== >>>>> epoch: 1559087699 date_epoch_local: 2019-05-28 19:54:59 utcepoch: >>>>> 1559088099 date_epoch_utc: 2019-05-28 23:54:59 tz: America/New_York >>>>> obsTimeUtc: 2019-05-28T23:54:59Z obsTimeLocal: 2019-05-28 19:54:59 >>>>> epoch: 1559087702 date_epoch_local: 2019-05-28 19:55:02 utcepoch: >>>>> 1559088102 date_epoch_utc: 2019-05-28 23:55:02 tz: America/New_York >>>>> obsTimeUtc: 2019-05-28T23:55:02Z obsTimeLocal: 2019-05-28 19:55:02 >>>>> Number of WU records: 240 >>>>> >>>>> ==> 0:20:28:5_wu.txt <== >>>>> epoch: 1559015699 date_epoch_local: 2019-05-27 23:54:59 utcepoch: >>>>> 1559016099 date_epoch_utc: 2019-05-28 03:54:59 tz: America/New_York >>>>> obsTimeUtc: 2019-05-28T03:54:59Z obsTimeLocal: 2019-05-27 23:54:59 >>>>> epoch: 1559015999 date_epoch_local: 2019-05-27 23:59:59 utcepoch: >>>>> 1559016399 date_epoch_utc: 2019-05-28 03:59:59 tz: America/New_York >>>>> obsTimeUtc: 2019-05-28T03:59:59Z obsTimeLocal: 2019-05-27 23:59:59 >>>>> Number of WU records: 288 >>>>> >>>>> >>>>> But after my system rolled over midnight localtime, results returned to >>>>> the correct dates when asking for 5-28: >>>>> >>>>> pi@nixie:/var/tmp $ head -3 0:0:29:5_wu.txt >>>>> Using configuration file /usr/share/weewx4/weewx.conf. >>>>> epoch: 1559016299 date_epoch_local: 2019-05-28 00:04:59 utcepoch: >>>>> 1559016699 date_epoch_utc: 2019-05-28 04:04:59 tz: America/New_York >>>>> obsTimeUtc: 2019-05-28T04:04:59Z obsTimeLocal: 2019-05-28 00:04:59 >>>>> epoch: 1559016599 date_epoch_local: 2019-05-28 00:09:59 utcepoch: >>>>> 1559016999 date_epoch_utc: 2019-05-28 04:09:59 tz: America/New_York >>>>> obsTimeUtc: 2019-05-28T04:09:59Z obsTimeLocal: 2019-05-28 00:09:59 >>>>> >>>>> pi@nixie:/var/tmp $ tail -3 0:0:29:5_wu.txt >>>>> epoch: 1559102099 date_epoch_local: 2019-05-28 23:54:59 utcepoch: >>>>> 1559102499 date_epoch_utc: 2019-05-29 03:54:59 tz: America/New_York >>>>> obsTimeUtc: 2019-05-29T03:54:59Z obsTimeLocal: 2019-05-28 23:54:59 >>>>> epoch: 1559102399 date_epoch_local: 2019-05-28 23:59:59 utcepoch: >>>>> 1559102799 date_epoch_utc: 2019-05-29 03:59:59 tz: America/New_York >>>>> obsTimeUtc: 2019-05-29T03:59:59Z obsTimeLocal: 2019-05-28 23:59:59 >>>>> Number of WU records: 288 >>>>> >>>>> >>>>> Regards, >>>>> \Leon >>>>> -- >>>>> Leon Shaner :: Dearborn, Michigan (iPad Pro) >>>>> >>>>> On May 29, 2019, at 8:53 PM, Rod Yager <[email protected]> wrote: >>>>> >>>>>> Further to this, it has now rolled past 10am here, so the local date is >>>>>> now the same as the UTC date. (ie. local time May 30 2019 10:40 AM = May >>>>>> 30 2019 00:40 AM UTC). >>>>>> >>>>>> Now I get: >>>>>> >>>>>> ./wunderfixer --verbose --date=2019-05-29 --epsilon=125 >>>>>> >>>>>> Using configuration file /home/weewx/weewx.conf. >>>>>> >>>>>> Using database binding 'wx_binding', which is bound to database >>>>>> 'archive_mysql' >>>>>> >>>>>> Weather Underground Station: xxxxx >>>>>> >>>>>> Date to check: 2019-05-29 >>>>>> >>>>>> Number of archive records: 288 >>>>>> >>>>>> Number of WU records: 288 >>>>>> >>>>>> Number of missing records: 0 >>>>>> >>>>>> [root@moses bin]# ./wunderfixer --verbose --date=2019-05-30 --epsilon=125 >>>>>> >>>>>> Using configuration file /home/weewx/weewx.conf. >>>>>> >>>>>> Using database binding 'wx_binding', which is bound to database >>>>>> 'archive_mysql' >>>>>> >>>>>> Weather Underground Station: xxxxxx >>>>>> >>>>>> Date to check: 2019-05-30 >>>>>> >>>>>> Number of archive records: 128 >>>>>> >>>>>> Number of WU records: 127 >>>>>> >>>>>> Number of missing records: 1 >>>>>> >>>>>> >>>>>> This means that WU is now actually providing the records for the date >>>>>> requested, rather than the day after the requested date. >>>>>> So it seems that what is happening is that WU is determining whether or >>>>>> not the current date at the station location is the same as the UTC date. >>>>>> If it is, it returns the data for the date as in the request. But if the >>>>>> local date is different, it makes an (unwanted) adjustment for the date >>>>>> difference. >>>>>> >>>>>> Rod >>>>>> >>>>>> >>>>>>> On Thursday, May 30, 2019 at 9:29:16 AM UTC+10, Leon Shaner wrote: >>>>>>> Hi, Rod, >>>>>>> >>>>>>> Yes and thanks for adding yet another confirmation of the issue. =D >>>>>>> >>>>>>> I can show that if I do the query within X hours of my offset of UTC, >>>>>>> what actually happens is they report 288 records from the day PRIOR to >>>>>>> the one I am asking about. >>>>>>> For example, I ask for 20190528 and they give me records for 20190527, >>>>>>> so *that* is why wunderfixer "thinks" it needs to re-upload everything. >>>>>>> >>>>>>> I am in contact with IBM about it and have shown them irrefutable proof >>>>>>> of the issue. >>>>>>> They didn't respond back yet, which I expect is because the proof was >>>>>>> irrefutable. Ha! ;-) >>>>>>> >>>>>>> I expect that they're investigating and would rather respond from a >>>>>>> position of understanding, or with any luck maybe even a quick fix. =D >>>>>>> >>>>>>> I meant to follow-up with IBM again this morning, but got waylaid, so >>>>>>> I'll do that now. >>>>>>> Thanks again, and for the reminder. =D >>>>>>> >>>>>>> Regards, >>>>>>> \Leon >>>>>>> -- >>>>>>> Leon Shaner :: Dearborn, Michigan (iPad Pro) >>>>>>> >>>>>>> On May 29, 2019, at 6:14 PM, Rod Yager <[email protected]> wrote: >>>>>>> >>>>>>>> There is definitely a time zone issue. I am in the Sydney Australia >>>>>>>> timezone (UTC +10 hours). >>>>>>>> >>>>>>>> It is currently 8am local time on May 30, 2019. (10pm May 29, 2019 >>>>>>>> UTC) >>>>>>>> >>>>>>>> If I execute >>>>>>>> >>>>>>>> ./wunderfixer --verbose --date=2019-05-29 --epsilon=125 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I get >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Using configuration file /home/weewx/weewx.conf. >>>>>>>> >>>>>>>> Using database binding 'wx_binding', which is bound to database >>>>>>>> 'archive_mysql' >>>>>>>> >>>>>>>> Weather Underground Station: xxxxxxx >>>>>>>> >>>>>>>> Date to check: 2019-05-29 >>>>>>>> >>>>>>>> Number of archive records: 288 >>>>>>>> >>>>>>>> Number of WU records: 97 >>>>>>>> >>>>>>>> >>>>>>>> Number of missing records: 288 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Now WU actually has 288 records for 2019-05-29. >>>>>>>> >>>>>>>> But it only has 97 records for 2019-05-30. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> So it is clear that wunderfixer is downloading the record data for >>>>>>>> 2019-05-30 from WeatherUnderground and trying to match them with the >>>>>>>> local records for 2019-30-29. >>>>>>>> >>>>>>>> Of course, they all mismatch, and so wunderfixer concludes that it >>>>>>>> must upload all the data for 2019-05-29. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hope this narrows down the search for a solution. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Rod >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Monday, May 27, 2019 at 9:35:25 PM UTC+10, Leon Shaner wrote: >>>>>>>>> >>>>>>>>> On May 27, 2019, at 12:12 AM, gjr80 <[email protected]> wrote: >>>>>>>>> >>>>>>>>>>> On Monday, 27 May 2019 13:16:53 UTC+10, Leon Shaner wrote: >>>>>>>>>>> [snip] >>>>>>>>>>> If you can see any shorter paths to a more reliable outcome than I >>>>>>>>>>> have achieved so far, then you know know know I will be very >>>>>>>>>>> grateful. =D >>>>>>>>>> >>>>>>>>>> I am not sure what local/UTC issue you refer to. When I do a >>>>>>>>>> api.weather.com/v2/pws/history query on a station to the east of >>>>>>>>>> Greenwich I am returned all records for the date specified (eg >>>>>>>>>> 20190525 gives me all records for 25 May 2019), each record contains >>>>>>>>>> an epoch timestamp which is correct and consistent with 25 May 2019. >>>>>>>>>> Everything is as I would expect. However, when I perform the same >>>>>>>>>> query on a station to the west of Greenwich I am returned records >>>>>>>>>> for the day before the date specified (ie 20190525 gives me all >>>>>>>>>> records for 24 May 2019 not 25 May 2019), again each record contains >>>>>>>>>> an epoch timestamp but the timestamp is for the previous day Ie 24 >>>>>>>>>> May 2019. I have checked a number of data records in the stations >>>>>>>>>> history table and WU is definitely returning the midnight to >>>>>>>>>> midnight data for the day before. I have confirmed this behaviour >>>>>>>>>> with a number of stations both east and west of Greenwich. >>>>>>>>>> >>>>>>>>>> I don't think there is a local/UTC time issue, I think WU is having >>>>>>>>>> some implementation issues and for stations west of Greenwich they >>>>>>>>>> are returning the wrong day of data. >>>>>>>>> >>>>>>>>> Thanks, Gary! This was all very helpful. >>>>>>>>> In addition to what you've described across the east vs west of GMT, >>>>>>>>> I get similar behavior if I am within X hours of my local UTC offset >>>>>>>>> when querying my own station. >>>>>>>>> Last night as soon as localtime rolled over midnight, the queries for >>>>>>>>> the previous day were correct. >>>>>>>>> >>>>>>>>> --Leon >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Gary >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "weewx-user" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>>>> an email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/weewx-user/683a28af-35e4-474e-95a0-f684b9926af0%40googlegroups.com. >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "weewx-user" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>> an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/weewx-user/3570f548-caf3-43e8-858d-1b4d8c87d84a%40googlegroups.com. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send an >>>>> email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/weewx-user/388426A6-12D0-45FC-AFF2-E6EB532B61DC%40isylum.org. >>>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "weewx-user" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an >>>> email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/weewx-user/184A035F-2BFD-4C43-9584-B84EB083F11F%40isylum.org. >>>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/D30A437D-57EF-46B0-A40F-98A7C1D2D793%40yager.net.au. >>> For more options, visit https://groups.google.com/d/optout. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/weewx-user/CB045A39-A0D2-415D-B8C2-B741AF0D263B%40isylum.org. >> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/BD67E5D8-2F65-477A-B7DB-AD0C5B984148%40yager.net.au. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/AB1F8DAE-A13C-4E1A-BA85-EFE4C6E19FF4%40isylum.org. For more options, visit https://groups.google.com/d/optout.
