Gary, It's late, so I'll respond to the rest later, but...
The problem here is that if we compare our local every 1-minute records to WU's query that only shows every 5-minute records, then we'll keep re-uploading the the "missing" 1-minute records every time wunderfixer is called. This amounts to pointless hits against WU's infrastructure, since pragmatically it's very clear they almost always drop all records not closely aligned to 5-minute "buckets." I'm trying to get to the heart of the matter re: what wunderfixer was really designed to do, which is to make sure what we have locally is consistent with what WU will actually REPORT, and when there are gaps in what WU *should* report, and we have local data to fill in those gaps, then re-upload. BUT to not gratuitously re-upload data that WU generally throws away (they just don't seem to care about storing data at < 5-minute intervals). Regards, \Leon -- Leon Shaner :: Dearborn, Michigan (iPad Pro) > On May 27, 2019, at 12:12 AM, gjr80 <gjroder...@gmail.com> wrote: > >> On Monday, 27 May 2019 13:16:53 UTC+10, Leon Shaner wrote: >> Gary, >> >> In practice, WU seems to discard data that is not close to their APPARENTly >> preferred 5-minute "normalization buckets." >> >> I upload via rapidfire *and* regular loop on 1-minute intervals, and >> irregardless of same, the queries only ever show the records most closely >> aligned to their "preferred" 5-minute "buckets." >> >> I would love to see them preserve the same interval that I upload, but >> either by design or omission or bug, they simply don't. :-( >> Could be the layers in-between are dropping data by code-exception. >> Could be it's deliberate. >> I'm not here to debug their code unless they want to start paying me a >> worthy salary. ;-) >> >> Well... MOST of the time the behavior I have described seems to be true. >> :-/ >> >> While WU normally SEEMS to only keeps records on 5-minute boundaries, I have >> seen where if wunderfixer is used persistently enough (say more than 10 >> times spread out every 20 minutes across a total span of 200 minutes) then >> it will keep records that are more frequent than every 5-minutes. >> >> Right now / especially lately, things are so sporadic with WU that I don't >> even want to spend cycles chasing what's really going on there. I would >> rather just only have wunderfixer ignore most records except those nearest >> the 5-minute boundaries that WU seems to most care about. >> > > Show Quoted Content >> Gary, >> >> In practice, WU seems to discard data that is not close to their APPARENTly >> preferred 5-minute "normalization buckets." >> >> I upload via rapidfire *and* regular loop on 1-minute intervals, and >> irregardless of same, the queries only ever show the records most closely >> aligned to their "preferred" 5-minute "buckets." >> >> I would love to see them preserve the same interval that I upload, but >> either by design or omission or bug, they simply don't. :-( >> Could be the layers in-between are dropping data by code-exception. >> Could be it's deliberate. >> I'm not here to debug their code unless they want to start paying me a >> worthy salary. ;-) >> >> Well... MOST of the time the behavior I have described seems to be true. >> :-/ >> >> While WU normally SEEMS to only keeps records on 5-minute boundaries, I have >> seen where if wunderfixer is used persistently enough (say more than 10 >> times spread out every 20 minutes across a total span of 200 minutes) then >> it will keep records that are more frequent than every 5-minutes. >> >> Right now / especially lately, things are so sporadic with WU that I don't >> even want to spend cycles chasing what's really going on there. I would >> rather just only have wunderfixer ignore most records except those nearest >> the 5-minute boundaries that WU seems to most care about. >> > I am not suggesting we/you/anyone should try to solve WU > issues/problems/whatever, merely making the point that WU is a blackbox, > nobody seems to really know what it does with the data that it is fed. If we > are going to have WU recreate/create 'missing' data then the logical approach > would be to feed it with the data it was orignally fed rather than trying to > guess what it wants. -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/8B0BFD31-7D91-4A3D-BDED-8923E8AF0C04%40isylum.org. For more options, visit https://groups.google.com/d/optout.
