So you're saying: - periodically loses connection to the station - yet when that happens it keeps writing out empty (your words) records every archive period - and when you press the button to reconnect to the station, you expect weewx to grab any historical records ?
The first one sounds like a hardware issue. The second one is a possible bug. The third is a feature request. That sound correct to you ? (I don't have a KLP so I can't speculate on how to fix your periodic loss of connectivity thing that starts the avalanche of issues, but I wanted to ask if I'm reading the problem description correctly) On Monday, January 4, 2021 at 1:26:03 PM UTC-8 [email protected] wrote: > not very long (except for the initial check - here are 51,330 records to > be covered) > > - the story is (I meanwhile checked the data base content and the syslog) > that weewx continues writing (empty) records when it isn't connected to the > console. > I think that's the reason for not reading the missed records as the is a > record (though empty) for the respective timestamps (archiving window = 1 > minute). > > Weewx should stop writing (empty records) into the database until it > receives the signal from the console again. Then check (like at the initial > start) > which historical records are missing, i.e. exist in the console memory but > not yet in the weewx database, retrieve and store them > and then continue with business as usual (archiving in the interval, > writing summary records, print charts ...) > > below the relevant syslog excerpts: > > I had weewx starting from scratch again - empty database - it was then > reading all available historical records in the console and stored > (archived) them > see below > ........................ > > Jan 4 14:20:50 raspberrypi weewx[29188] INFO user.kl: handleHistoryData: > records cached=1766, records skipped=7, next=38341.0 > Jan 4 14:20:50 raspberrypi weewx[29188] INFO user.kl: handleHistoryData: > records cached=1772, records skipped=7, next=38347.0 > Jan 4 14:20:51 raspberrypi weewx[29188] INFO user.kl: handleHistoryData: > records cached=1778, records skipped=7, next=38353.0 > Jan 4 14:20:51 raspberrypi weewx[29188] INFO user.kl: handleHistoryData: > records cached=1784, records skipped=7, next=38359.0 > Jan 4 14:20:51 raspberrypi weewx[29188] INFO user.kl: handleHistoryData: > records cached=1790, records skipped=7, next=38365.0 > Jan 4 14:20:52 raspberrypi weewx[29188] INFO user.kl: handleHistoryData: > records cached=1796, records skipped=7, next=38371.0 > Jan 4 14:21:01 raspberrypi weewx[29188] INFO user.kl: Records scanned: > 1800 > Jan 4 14:21:01 raspberrypi weewx[29188] INFO user.kl: Found 1799 > historical records > Jan 4 14:21:01 raspberrypi weewx[29188] INFO user.kl: Handle record > 46801: 2021-01-01 11:00:00 CET (1609495200) > Jan 4 14:21:01 raspberrypi weewx[29188] INFO user.kl: Handle record > 46802: 2021-01-01 11:01:00 CET (1609495260) > Jan 4 14:21:01 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-01 11:01:00 CET (1609495260) to database 'weewx-kl.sdb' > Jan 4 14:21:01 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-01 11:01:00 CET (1609495260) to daily summary in 'weewx-kl.sdb' > Jan 4 14:21:01 raspberrypi weewx[29188] INFO user.kl: Handle record > 46803: 2021-01-01 11:02:00 CET (1609495320) > Jan 4 14:21:01 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-01 11:02:00 CET (1609495320) to database 'weewx-kl.sdb' > Jan 4 14:21:01 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-01 11:02:00 CET (1609495320) to daily summary in 'weewx-kl.sdb' > Jan 4 14:21:01 raspberrypi weewx[29188] INFO user.kl: Handle record > 46804: 2021-01-01 11:03:00 CET (1609495380) > > ................. > > now the loading comes to an end and weewx runs in normal mode (i.e. > reading not history but actual data) > > ................. > > Jan 4 14:33:50 raspberrypi weewx[29188] INFO user.kl: Handle record > 51329: 2021-01-04 14:31:00 CET (1609767060) > Jan 4 14:33:50 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 14:31:00 CET (1609767060) to database 'weewx-kl.sdb' > Jan 4 14:33:50 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 14:31:00 CET (1609767060) to daily summary in 'weewx-kl.sdb' > Jan 4 14:33:50 raspberrypi weewx[29188] INFO user.kl: *Handle record > 51330:* 2021-01-04 14:32:00 CET (1609767120) > Jan 4 14:33:50 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 14:32:00 CET (1609767120) to database 'weewx-kl.sdb' > Jan 4 14:33:50 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 14:32:00 CET (1609767120) to daily summary in 'weewx-kl.sdb' > Jan 4 14:33:50 raspberrypi weewx[29188] INFO user.kl: *Saved 929 > historical records; ts last saved record 2021-01-04 14:32:00 CET > (1609767120)* > Jan 4 14:33:50 raspberrypi weewx[29188] INFO *weewx.engine: Starting > main packet loop.* > Jan 4 14:34:20 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 14:34:00 CET (1609767240) to database 'weewx-kl.sdb' > Jan 4 14:34:20 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 14:34:00 CET (1609767240) to daily summary in 'weewx-kl.sdb' > Jan 4 14:34:20 raspberrypi weewx[29188] INFO weewx.cheetahgenerator: > Generated 1 files for report kl in 0.28 seconds > Jan 4 14:34:29 raspberrypi weewx[29188] INFO weewx.imagegenerator: > Generated 33 images for report kl in 8.42 seconds > Jan 4 14:35:30 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 14:35:00 CET (1609767300) to database 'weewx-kl.sdb' > Jan 4 14:35:30 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 14:35:00 CET (1609767300) to daily summary in 'weewx-kl.sdb' > .................... > Jan 4 14:58:25 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 14:58:00 CET (1609768680) to database 'weewx-kl.sdb' > Jan 4 14:58:25 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 14:58:00 CET (1609768680) to daily summary in 'weewx-kl.sdb' > Jan 4 14:58:25 raspberrypi weewx[29188] INFO weewx.cheetahgenerator: > Generated 1 files for report kl in 0.13 seconds > Jan 4 14:58:26 raspberrypi weewx[29188] INFO weewx.imagegenerator: > Generated 11 images for report kl in 0.70 seconds > Jan 4 14:59:15 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 14:59:00 CET (1609768740) to database 'weewx-kl.sdb' > Jan 4 14:59:15 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 14:59:00 CET (1609768740) to daily summary in 'weewx-kl.sdb' > Jan 4 14:59:15 raspberrypi weewx[29188] INFO weewx.cheetahgenerator: > Generated 1 files for report kl in 0.13 seconds > Jan 4 14:59:16 raspberrypi weewx[29188] INFO weewx.imagegenerator: > Generated 11 images for report kl in 0.73 seconds > Jan 4 15:00:15 raspberrypi weewx[29188] INFO > *weewx.manager: Added record 2021-01-04 15:00:00 CET (1609768800) to > database 'weewx-kl.sdb' ***************** here the first empty record is > written as one see in the database *********** > Jan 4 15:00:15 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 15:00:00 CET (1609768800) to daily summary in 'weewx-kl.sdb' > Jan 4 15:00:15 raspberrypi weewx[29188] INFO weewx.cheetahgenerator: > Generated 1 files for report kl in 0.16 seconds > Jan 4 15:00:24 raspberrypi weewx[29188] INFO weewx.imagegenerator: > Generated 33 images for report kl in 8.42 seconds > Jan 4 15:01:15 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 15:01:00 CET (1609768860) to database 'weewx-kl.sdb' > .................. > Jan 4 15:17:18 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 15:17:00 CET (1609769820) to database 'weewx-kl.sdb' > Jan 4 15:17:18 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 15:17:00 CET (1609769820) to daily summary in 'weewx-kl.sdb' > Jan 4 15:17:19 raspberrypi weewx[29188] INFO weewx.cheetahgenerator: > Generated 1 files for report kl in 0.13 seconds > Jan 4 15:17:19 raspberrypi weewx[29188] INFO weewx.imagegenerator: > Generated 11 images for report kl in 0.64 seconds > Jan 4 15:18:18 raspberrypi weewx[29188] INFO *user.kl: no new weather > data after 1209 seconds* > Jan 4 15:18:18 raspberrypi weewx[29188] INFO > *user.kl: no contact with console after 1209 seconds (press the USB button > to start communication) ************* of course it takes some time until > you realize that the connection is interrupted and you have to press this > USB button *********** > Jan 4 15:18:18 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 15:18:00 CET (1609769880) to database 'weewx-kl.sdb' > Jan 4 15:18:18 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 15:18:00 CET (1609769880) to daily summary in 'weewx-kl.sdb' > ........................... > Jan 4 21:19:28 raspberrypi weewx[29188] INFO *user.kl: no new weather > data after 22879 seconds* > Jan 4 21:19:28 raspberrypi weewx[29188] INFO user.kl: no contact with > console after 22879 seconds (press the USB button to start communication) > Jan 4 21:20:18 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:20:00 CET (1609791600) to database 'weewx-kl.sdb' > Jan 4 21:20:18 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:20:00 CET (1609791600) to daily summary in 'weewx-kl.sdb' > Jan 4 21:20:19 raspberrypi weewx[29188] INFO weewx.cheetahgenerator: > Generated 1 files for report kl in 0.16 seconds > Jan 4 21:20:19 raspberrypi weewx[29188] INFO weewx.imagegenerator: > Generated 11 images for report kl in 0.63 seconds > Jan 4 21:21:19 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:21:00 CET (1609791660) to database 'weewx-kl.sdb' > Jan 4 21:21:19 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:21:00 CET (1609791660) to daily summary in 'weewx-kl.sdb' > Jan 4 21:21:19 raspberrypi weewx[29188] INFO weewx.cheetahgenerator: > Generated 1 files for report kl in 0.13 seconds > Jan 4 21:21:19 raspberrypi weewx[29188] INFO weewx.imagegenerator: > Generated 11 images for report kl in 0.63 seconds > Jan 4 21:22:19 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:22:00 CET (1609791720) to database 'weewx-kl.sdb' > Jan 4 21:22:19 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:22:00 CET (1609791720) to daily summary in 'weewx-kl.sdb' > Jan 4 21:22:19 raspberrypi weewx[29188] INFO weewx.cheetahgenerator: > Generated 1 files for report kl in 0.14 seconds > Jan 4 21:22:20 raspberrypi weewx[29188] INFO weewx.imagegenerator: > Generated 11 images for report kl in 0.67 seconds > Jan 4 21:23:19 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:23:00 CET (1609791780) to database 'weewx-kl.sdb' > Jan 4 21:23:19 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:23:00 CET (1609791780) to daily summary in 'weewx-kl.sdb' > Jan 4 21:23:19 raspberrypi weewx[29188] INFO weewx.cheetahgenerator: > Generated 1 files for report kl in 0.13 seconds > Jan 4 21:23:20 raspberrypi weewx[29188] INFO weewx.imagegenerator: > Generated 11 images for report kl in 0.62 seconds > Jan 4 21:24:19 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:24:00 CET (1609791840) to database 'weewx-kl.sdb' > Jan 4 21:24:19 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:24:00 CET (1609791840) to daily summary in 'weewx-kl.sdb' > Jan 4 21:24:19 raspberrypi weewx[29188] INFO weewx.cheetahgenerator: > Generated 1 files for report kl in 0.16 seconds > Jan 4 21:24:20 raspberrypi weewx[29188] INFO weewx.imagegenerator: > Generated 11 images for report kl in 0.62 seconds > Jan 4 21:25:19 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:25:00 CET (1609791900) to database 'weewx-kl.sdb' > Jan 4 21:25:19 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:25:00 CET (1609791900) to daily summary in 'weewx-kl.sdb' > Jan 4 21:25:20 raspberrypi weewx[29188] INFO weewx.cheetahgenerator: > Generated 1 files for report kl in 0.13 seconds > Jan 4 21:25:21 raspberrypi weewx[29188] INFO weewx.imagegenerator: > Generated 11 images for report kl in 0.67 seconds > Jan 4 21:25:30 raspberrypi vncserver-x11[472,root]: Connections: > connected: 192.168.1.88::60035 (TCP) > Jan 4 21:25:30 raspberrypi vncserver-x11[472,root]: Connections: > authenticated: 192.168.1.88::60035 (TCP), as pi (f permissions) > Jan 4 21:25:30 raspberrypi colord[495]: failed to get session [pid 5510]: > No data available > > > *Jan 4 21:26:20 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:26:00 CET (1609791960) to database 'weewx-kl.sdb' ********** > missing connection discovered, USB button pressed - first non-empty record > written - however no history since interruption (even though the respective > data resides in the console - the trouble is that weewx wrote these empty > records ************ > Jan 4 21:26:20 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:26:00 CET (1609791960) to daily summary in 'weewx-kl.sdb' > Jan 4 21:26:20 raspberrypi weewx[29188] INFO weewx.cheetahgenerator: > Generated 1 files for report kl in 0.13 seconds > Jan 4 21:26:21 raspberrypi weewx[29188] INFO weewx.imagegenerator: > Generated 11 images for report kl in 0.62 seconds > Jan 4 21:27:20 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:27:00 CET (1609792020) to database 'weewx-kl.sdb' > Jan 4 21:27:20 raspberrypi weewx[29188] INFO weewx.manager: Added record > 2021-01-04 21:27:00 CET (1609792020) to daily summary in 'weewx-kl.sdb' > > > On 04.01.2021 21:42, vince wrote: > > How long does this check take when weewx restarts ? How much data do you > think is being missed ? > > We'd have to see some logs. Set debug=1 and restart weewx and let it run > for a couple archive periods. > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/fe55c5e1-0b37-410f-94bb-9f68c3a2db5dn%40googlegroups.com > > <https://groups.google.com/d/msgid/weewx-user/fe55c5e1-0b37-410f-94bb-9f68c3a2db5dn%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/2a8dfdea-8934-4080-96e5-79526d9d16fcn%40googlegroups.com.
