I am not seeing the log entries from the instrumented code that I included. It will tell us much more about what is going on. Please include it.
For the record, I was also able to rebuild the daily summaries uneventfully using your dumped archive table. But, it occurred to me, midnight here on the west coast of the USA (UTC-7) is a different time zone from you, which I assume is CEST. I set up a VM using CEST but, again, it rebuilt the summaries uneventfully. Really need to see the log. -tk On Wed, Dec 25, 2024 at 2:40 PM mihec <[email protected]> wrote: > Thanks for checking. I ran the command again and monitored memfree but it > was nothing to be worried about. > Here is how I ran the command (note the from-to and where the processing > was when I captured this status): > > username@hostname:~$ sudo weectl database rebuild-daily --from=2021-01-01 > --to=2022-01-01 > Using configuration file /etc/weewx/weewx.conf > Daily summaries from 2021-01-01 through 2022-01-01, inclusive, will be > rebuilt. > Rebuild the daily summaries in the database 'weewx.sdb' (y/n)? y > Rebuilding daily summaries in database 'weewx.sdb' ... > Daily summaries at V2.0. Patching to V4.0 > Processed through: 2020-08-28 00:00:00 CEST (1598565600) > > It again fails at the same point. I'm sorry to bother you so much with > this. > I just tried on a laptop where I run weewx 4.10. The 'wee_database > --rebuild-daily' fails with same error at same point as on the other two > systems. > For the record, the RPi is 3B+ running bullseye 32-bit, my desktop is old > i7-6700 3.4GHz with 8 GB, running Ubuntu 22.04.1. Laptop is i7-7500 2.70 > GHz with 8 GB running Ubuntu 22.04.1. > > > sreda, 25. december 2024 ob 23:12:39 UTC+1 je oseba Tom Keffer napisala: > >> I was thinking the same. >> >> On Wed, Dec 25, 2024 at 2:10 PM vince <[email protected]> wrote: >> >>> Thank you. I built a db from it and ran rebuild-daily and it worked >>> just fine with no errors in 2.5 minutes. No errors running database >>> check. No errors running database update. >>> >>> Computer was a m1 mac mini with 16 GB of RAM. I wonder if you're >>> running out of memory and need to split your rebuild-daily up into pieces >>> with the --from and --to options. Just a guess. >>> >>> >>> On Wednesday, December 25, 2024 at 1:56:50 PM UTC-8 mihec wrote: >>> >>>> This is the dump: https://we.tl/t-X3r6gbimBx >>>> >>>> sreda, 25. december 2024 ob 22:48:59 UTC+1 je oseba mihec napisala: >>>> >>>>> I replaced the original manager.py with attached one. That file is in >>>>> the /usr/share/weewx/weewx/ directory. >>>>> I will prepare the dump of the database. >>>>> >>>>> username@hostname:~$ sudo weectl database rebuild-daily >>>>> >>>>> Using configuration file /etc/weewx/weewx.conf >>>>> All daily summaries will be rebuilt. >>>>> Rebuild the daily summaries in the database 'weewx.sdb' (y/n)? y >>>>> Rebuilding daily summaries in database 'weewx.sdb' ... >>>>> Daily summaries at V2.0. Patching to V4.0 >>>>> Traceback (most recent call last):0:00 CEST (1628719200) >>>>> File "/usr/share/weewx/weedb/sqlite.py", line 38, in guarded_fn >>>>> return fn(*args, **kwargs) >>>>> File "/usr/share/weewx/weedb/sqlite.py", line 233, in execute >>>>> return sqlite3.Cursor.execute(self, *args, **kwargs) >>>>> sqlite3.OperationalError: no such column: inf >>>>> >>>>> During handling of the above exception, another exception occurred: >>>>> >>>>> Traceback (most recent call last): >>>>> File "/usr/share/weewx/weectl.py", line 75, in <module> >>>>> main() >>>>> File "/usr/share/weewx/weectl.py", line 67, in main >>>>> namespace.func(namespace) >>>>> File "/usr/share/weewx/weectllib/__init__.py", line 90, in dispatch >>>>> namespace.action_func(config_dict, namespace) >>>>> File "/usr/share/weewx/weectllib/database_cmd.py", line 329, in >>>>> rebuild_daily >>>>> weectllib.database_actions.rebuild_daily(config_dict, >>>>> File "/usr/share/weewx/weectllib/database_actions.py", line 134, in >>>>> rebuild_daily >>>>> with weewx.manager.open_manager(manager_dict, initialize=True) as >>>>> dbm: >>>>> File "/usr/share/weewx/weewx/manager.py", line 924, in open_manager >>>>> return manager_cls.open_with_create(manager_dict['database_dict'], >>>>> File "/usr/share/weewx/weewx/manager.py", line 244, in >>>>> open_with_create >>>>> dbmanager = cls(connection, table_name=table_name, schema=schema) >>>>> File "/usr/share/weewx/weewx/manager.py", line 1076, in __init__ >>>>> self.patch_sums() >>>>> File "/usr/share/weewx/weewx/manager.py", line 1537, in patch_sums >>>>> >>>>> self.recalculate_weights(start_d=datetime.date(2020, 6, 1)) >>>>> File "/usr/share/weewx/weewx/manager.py", line 1451, in >>>>> recalculate_weights >>>>> self._do_tranche(mark_d, end_of_tranche_d, weight_fn, progress_fn) >>>>> File "/usr/share/weewx/weewx/manager.py", line 1490, in _do_tranche >>>>> self._set_day_sums(day_accum, cursor) >>>>> File "/usr/share/weewx/weewx/manager.py", line 1516, in _set_day_sums >>>>> >>>>> cursor.execute(update_sql, (day_accum.timespan.start,)) >>>>> File "/usr/share/weewx/weedb/sqlite.py", line 50, in guarded_fn >>>>> raise weedb.NoColumnError(e) >>>>> weedb.NoColumnError: no such column: inf >>>>> >>>>> >>>>> sreda, 25. december 2024 ob 21:13:17 UTC+1 je oseba Tom Keffer >>>>> napisala: >>>>> >>>>>> Thanks. That tells me what I needed to know to create a useful >>>>>> instrumented version of manager.py. Replace the one you have with the >>>>>> attached version. It will give more information about exactly what >>>>>> created >>>>>> the error. Post the log. >>>>>> >>>>>> -tk >>>>>> >>>>>> On Wed, Dec 25, 2024 at 9:29 AM mihec <[email protected]> wrote: >>>>>> >>>>>>> Hi, first I want to wish you Merry Christmas and Happy Holidays! >>>>>>> >>>>>>> This is the output of the "weectl database rebuild-daily". Please >>>>>>> note that I've copied the weewx.sdb database file to a desktop PC to >>>>>>> make >>>>>>> it run much faster. I've replaced the real username and hostname with >>>>>>> something generic. The weewx's version is same as in RPi. >>>>>>> >>>>>>> When executed, the 'rebuild' process starts around June 2020 and >>>>>>> goes on normally until it fails on August 12, 2021 (epoch >>>>>>> 1628719200). In the output below, you only see the status when it >>>>>>> fails already. I can provide with a screenshot if that helps. >>>>>>> The thing is that any command on the database (update, check or >>>>>>> rebuild) fails on the same date (record?). >>>>>>> >>>>>>> username@host:~$ ls -la weewx.sdb >>>>>>> -rwxrwxr-x 1 username username 682086400 dec 25 18:18 weewx.sdb >>>>>>> username@host:~$ weewxd -v >>>>>>> 5.1.0 >>>>>>> username@host:~$ weectl -v >>>>>>> weectl 5.1.0 >>>>>>> username@host:~$ sudo weectl database rebuild-daily >>>>>>> [sudo] password for username: >>>>>>> Using configuration file /etc/weewx/weewx.conf >>>>>>> All daily summaries will be rebuilt. >>>>>>> Rebuild the daily summaries in the database 'weewx.sdb' (y/n)? y >>>>>>> Rebuilding daily summaries in database 'weewx.sdb' ... >>>>>>> Daily summaries at V2.0. Patching to V4.0 >>>>>>> Traceback (most recent call last):0:00 CEST (1628719200) >>>>>>> File "/usr/share/weewx/weedb/sqlite.py", line 38, in guarded_fn >>>>>>> return fn(*args, **kwargs) >>>>>>> File "/usr/share/weewx/weedb/sqlite.py", line 233, in execute >>>>>>> return sqlite3.Cursor.execute(self, *args, **kwargs) >>>>>>> sqlite3.OperationalError: no such column: inf >>>>>>> >>>>>>> During handling of the above exception, another exception occurred: >>>>>>> >>>>>>> Traceback (most recent call last): >>>>>>> File "/usr/share/weewx/weectl.py", line 75, in <module> >>>>>>> main() >>>>>>> File "/usr/share/weewx/weectl.py", line 67, in main >>>>>>> namespace.func(namespace) >>>>>>> File "/usr/share/weewx/weectllib/__init__.py", line 90, in dispatch >>>>>>> namespace.action_func(config_dict, namespace) >>>>>>> File "/usr/share/weewx/weectllib/database_cmd.py", line 329, in >>>>>>> rebuild_daily >>>>>>> weectllib.database_actions.rebuild_daily(config_dict, >>>>>>> File "/usr/share/weewx/weectllib/database_actions.py", line 134, >>>>>>> in rebuild_daily >>>>>>> with weewx.manager.open_manager(manager_dict, initialize=True) >>>>>>> as dbm: >>>>>>> File "/usr/share/weewx/weewx/manager.py", line 924, in open_manager >>>>>>> return >>>>>>> manager_cls.open_with_create(manager_dict['database_dict'], >>>>>>> File "/usr/share/weewx/weewx/manager.py", line 244, in >>>>>>> open_with_create >>>>>>> dbmanager = cls(connection, table_name=table_name, schema=schema) >>>>>>> File "/usr/share/weewx/weewx/manager.py", line 1076, in __init__ >>>>>>> self.patch_sums() >>>>>>> File "/usr/share/weewx/weewx/manager.py", line 1531, in patch_sums >>>>>>> self.recalculate_weights(start_d=datetime.date(2020, 6, 1)) >>>>>>> File "/usr/share/weewx/weewx/manager.py", line 1451, in >>>>>>> recalculate_weights >>>>>>> self._do_tranche(mark_d, end_of_tranche_d, weight_fn, >>>>>>> progress_fn) >>>>>>> File "/usr/share/weewx/weewx/manager.py", line 1490, in _do_tranche >>>>>>> self._set_day_sums(day_accum, cursor) >>>>>>> File "/usr/share/weewx/weewx/manager.py", line 1515, in >>>>>>> _set_day_sums >>>>>>> cursor.execute(update_sql, (day_accum.timespan.start,)) >>>>>>> File "/usr/share/weewx/weedb/sqlite.py", line 50, in guarded_fn >>>>>>> raise weedb.NoColumnError(e) >>>>>>> weedb.NoColumnError: no such column: inf >>>>>>> username@host:~$ >>>>>>> >>>>>>> torek, 24. december 2024 ob 15:15:16 UTC+1 je oseba Tom Keffer >>>>>>> napisala: >>>>>>> >>>>>>>> Let's put aside the database update and just focus on rebuilding >>>>>>>> the daily summaries. >>>>>>>> >>>>>>>> Could you please include the stack trace when doing "weectl >>>>>>>> database rebuild-daily"? Once I see where it fails, I can send you an >>>>>>>> instrumented version of the database code so we can see exactly what is >>>>>>>> happening. >>>>>>>> >>>>>>>> -tk >>>>>>>> >>>>>>>> On Tue, Dec 24, 2024 at 3:39 AM mihec <[email protected]> wrote: >>>>>>>> >>>>>>>>> Well, I've checked also the weectl database check which fails at >>>>>>>>> the same point as the update. >>>>>>>>> >>>>>>>>> >>>>>>>>> torek, 24. december 2024 ob 12:31:36 UTC+1 je oseba Auchtermuchty >>>>>>>>> Weather napisala: >>>>>>>>> >>>>>>>>>> That's not what it does now. >>>>>>>>>> >>>>>>>>>> " Databases created earlier than 3.7.0 (released 11-March-2017) >>>>>>>>>> have a flaw that prevents them from being used with archive >>>>>>>>>> intervals that >>>>>>>>>> change with time. This utility check whether your database is >>>>>>>>>> affected." >>>>>>>>>> >>>>>>>>>> $ weectl database check >>>>>>>>>> Using configuration file /etc/weewx/weewx.conf >>>>>>>>>> Checking daily summary tables version... >>>>>>>>>> Daily summary tables are at version 4.0. >>>>>>>>>> Interval Weighting Fix is not required. >>>>>>>>>> $ >>>>>>>>>> >>>>>>>>>> On Tuesday, 24 December 2024 at 11:13:48 UTC Graham Eddy wrote: >>>>>>>>>> >>>>>>>>>>> weewx used to include a tool for picking up things like a string >>>>>>>>>>> where a number should be. maybe `weectl database check’ still does >>>>>>>>>>> this >>>>>>>>>>> *⊣GE⊢* >>>>>>>>>>> >>>>>>>>>>> On 24 Dec 2024, at 4:17 pm, mihec <[email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>> I think there must be an invalid entry in my database (Aug 12, >>>>>>>>>>> 2021). My SQLite database management knowledge is basically zero, >>>>>>>>>>> so I'm >>>>>>>>>>> looking for advice >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "weewx-user" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> >>>>>>>> To view this discussion visit >>>>>>>>> https://groups.google.com/d/msgid/weewx-user/e60483cc-3910-45f8-aff4-7b4e95939cd9n%40googlegroups.com >>>>>>>>> <https://groups.google.com/d/msgid/weewx-user/e60483cc-3910-45f8-aff4-7b4e95939cd9n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "weewx-user" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> >>>>>> To view this discussion visit >>>>>>> https://groups.google.com/d/msgid/weewx-user/e92cf44c-d06f-4de1-ba04-440bceed755fn%40googlegroups.com >>>>>>> <https://groups.google.com/d/msgid/weewx-user/e92cf44c-d06f-4de1-ba04-440bceed755fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >> To view this discussion visit >>> https://groups.google.com/d/msgid/weewx-user/124fd290-b296-4c56-a4d4-43e7b30ae2d7n%40googlegroups.com >>> <https://groups.google.com/d/msgid/weewx-user/124fd290-b296-4c56-a4d4-43e7b30ae2d7n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion visit > https://groups.google.com/d/msgid/weewx-user/8fe396c2-d6b2-4aac-a8c8-fc014e9d2df8n%40googlegroups.com > <https://groups.google.com/d/msgid/weewx-user/8fe396c2-d6b2-4aac-a8c8-fc014e9d2df8n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEDMNaiCezWAmoUrn7kihFjCQ3b168Nj01Ex2s22Rr8s%3DQ%40mail.gmail.com.
