Hello, I have succesfully imported Weathercat data from about 7 years. That was about year and half ago. I remember, that I didn't have any problems with daylight saving. And In Finland we have summer time , which is +1 hour.
I have data logged in 1 minute intervals, and I used X minutes option in import config. torstai 25. elokuuta 2022 klo 16.54.11 UTC+3 kurand kirjoitti: > Thanks for confirming what I understood from weeimport.py and > weathercatimport.py. > > I am using "derived' for the import because I originally configured > WeatherCat with Adaptive interval set true so I have data logged at > varying rates from 1 minute all the way up to 30+ minute intervals. > > I think in this case, weathercatimport.py needs to use timezone aware > date-time objects during its construction of dateTime else data will become > misplaced and DST changes will fail as noted. Once this is accomplished, > then something needs to get added to deal with the apparent bug in > WeatherCat where transitioning from standard time to daylight saving time > in spring causes it to skip an extra hour forward sometimes. It looks to me > like this can all be isolated in weathercatimport.py's getRawData function. > > I'm curious to know if anyone else has run into this problem, or if it is > something specific to my WeatherCat data. Or perhaps WeatherCat import has > been used in anger. > > On Thursday, August 25, 2022 at 11:59:34 AM UTC+1 gjr80 wrote: > >> If you are using 'derive' to derive the archive interval from the >> imported data then the imported records must be in ascending >> timestamp/date-time order. This is because the derived archive interval is >> calculated by taking the difference between the timestamp of the current >> record and the timestamp of the previous record. If the calculated interval >> is negative it indicates the records being imported are not in timestamp >> order and the calculated archive interval cannot be trusted. The import >> will abort. >> >> This should not present a problem at the start of a daylight saving >> period where clocks jump forward (though the archive interval would be >> incorrectly calculated for the first record after daylight saving begins). >> It would certainly cause a problem at the end of a period of daylight >> saving where clocks jump back in time. >> >> So what to do? WeeWX date-time objects are timezone naive >> <https://docs.python.org/3/library/datetime.html#aware-and-naive-objects> >> so managing daylight saving is near impossible. I would suggest removing >> the data to be imported with the duplicate date-times (which set of >> duplicates I leave up to you) and re-trying the import. You might also want >> to consider using a fixed interval (if your data does indeed have a fixed >> interval) rather than deriving it from the data. FWIW, the vantage driver >> has a similar issue in that each year when daylight saving ends the driver >> receives archive records with duplicate timestamps, in this case the >> situation is handled slightly more gracefully in that WeeWX ignores the >> duplicate timestamped records with a 'PRIMARY KEY must be unique' error. >> wee_import is somewhat less forgiving. >> >> Gary >> On Thursday, 25 August 2022 at 00:45:13 UTC+10 kurand wrote: >> >>> Has anyone successfully imported data from WeatherCat for months where >>> Daylight Savings Time starts or ends? >>> >>> When I have tried this, I get the following type of error (example for >>> Oct-2017): >>> >>> Cannot derive 'interval' for record timestamp: 2017-10-29 01:00:00 BST >>> (1509235200). >>> >>> **** Raw data is not in ascending date time order. >>> >>> Looking at the 7 years of data I have logged by WeatherCat, it appears >>> that WeatherCat does not really handle DST transitions very well. Data is >>> generally logging in local time. >>> >>> For spring DST changes (in March for my UK based timezone), it seems >>> usually to jump forward by two hours at 0100 going to 0300 rather than just >>> one hour as you'd expect. It then tack on an hours worth of data for day 01 >>> at the end of the month. This gives rise to the above error when the end of >>> the month's data is processed. However, for some reason, in my data for >>> 2014, it does only jump forward one hour as expected and there was no extra >>> data logged at the end of the month. >>> >>> For autumn DST changes (in October in UK timezone), it seems to jump >>> back an hour at 0200, so it repeats log entries for hour 0100 again. This >>> gives rise to the above error. >>> >>> It's all a bit of a mess really; I'm not sure if its just my WeatherCat >>> data, or the way I configured it with Adaptive interval set true or >>> something else. >>> >>> Does anyone have a fix for this, or do I need to work something out. >>> >> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/4df8a3ac-6e22-4120-9fc6-de2889f2a58cn%40googlegroups.com.
