Got it, and I'm in the middle of running the calc-missing step right now. But in reviewing the describe archive outputs from the old and the new database, I see that dateTime is set as a primary key in the new table and not in the old one. Am I wrong in thinking that's also likely to impact performance.
On Sunday, June 28, 2026 at 2:42:09 PM UTC-4 Tom Keffer wrote: > There is no reason to migrate to the new schema. It is not an "upgrade." > It just offers more types. In your case, because you need only a single new > type (appTemp) it would be much more efficient to just add it using the > tool "weectl database add-column > <https://www.weewx.com/docs/5.4/utilities/weectl-database/#add-a-new-observation-type-to-the-database> > ": > > *weectl database add-column appTemp* > > > However, while that will add a new column, it will not populate it with > anything. To do that, use weectl database calc-missing > <https://www.weewx.com/docs/5.4/utilities/weectl-database/#calculate-missing-derived-variables> > : > > *weectl database calc-missing* > > > I would advise doing a backup first. > > -tk > > On Sun, Jun 28, 2026 at 11:25 AM [email protected] <[email protected]> > wrote: > >> Hmmm. My MySQL database is using the old schema, and it seems there's a >> good likelihood that's at least contributing to the performance problems >> I'm seeing. And while the link Tom gave describes adding a specific column >> to the table, >> https://weewx.com/docs/5.4/custom/database/#reconfigure-using-new-schema >> provides instructions that would seem to address migrating to the new >> schema completely. So I tried them. Stopped weewx, made a backup of the >> mysql database using mysqldump, and edited weewx.conf to revert to using >> the mysql database, and to use the wview_extended schema. Then ran weectl >> database reconfigure--which died after just under a minute: >> dan@wxbox ➜ ~ time sudo -u weewx weectl database reconfigure >> Using configuration file /etc/weewx/weewx.conf >> Copying database 'weewx' to 'weewx_new' >> The new database will use the same unit system as the old ('US'). >> Are you sure you wish to proceed (y/n)? y >> [2] 7144 killed sudo -u weewx weectl database reconfigure >> sudo -u weewx weectl database reconfigure 0.01s user 0.01s system 0% cpu >> 53.676 total >> >> The new database is created, the new archive table is created in it, and >> it appears to have all the necessary fields (describe archive returns 115 >> rows), but it's empty. The system log doesn't report any errors: >> 2026-06-28T14:16:26.783311-04:00 wxbox weectl[7147]: INFO weectllib: >> Initializing weectl version 5.4.0 >> 2026-06-28T14:16:26.787432-04:00 wxbox weectl[7147]: INFO weectllib: >> Command line: /usr/share/weewx/weectl.py database reconfigure >> 2026-06-28T14:16:26.788797-04:00 wxbox weectl[7147]: INFO weectllib: >> Using Python: 3.13.5 (main, May 5 2026, 21:05:52) [GCC 14.2.0] >> 2026-06-28T14:16:26.788976-04:00 wxbox weectl[7147]: INFO weectllib: >> Located at: /usr/bin/python3 >> 2026-06-28T14:16:26.789025-04:00 wxbox weectl[7147]: INFO weectllib: >> Platform: Linux-6.12.94+deb13-amd64-x86_64-with-glibc2.41 >> 2026-06-28T14:16:26.789078-04:00 wxbox weectl[7147]: INFO weectllib: >> Locale: 'en_US.UTF-8' >> 2026-06-28T14:16:26.789130-04:00 wxbox weectl[7147]: INFO weectllib: >> Entry path: /usr/share/weewx/weectl.py >> 2026-06-28T14:16:26.789303-04:00 wxbox weectl[7147]: INFO weectllib: >> WEEWX_ROOT: /etc/weewx >> 2026-06-28T14:16:26.789712-04:00 wxbox weectl[7147]: INFO weectllib: >> Config file: /etc/weewx/weewx.conf >> 2026-06-28T14:16:26.789764-04:00 wxbox weectl[7147]: INFO weectllib: User >> module: /etc/weewx/bin/user >> 2026-06-28T14:16:26.789957-04:00 wxbox weectl[7147]: INFO weectllib: >> Debug: 1 >> 2026-06-28T14:16:26.790009-04:00 wxbox weectl[7147]: INFO weectllib: >> User: weewx >> 2026-06-28T14:16:26.790050-04:00 wxbox weectl[7147]: INFO weectllib: >> Group: weewx >> 2026-06-28T14:16:26.790186-04:00 wxbox weectl[7147]: INFO weectllib: >> Groups: dialout weewx >> 2026-06-28T14:16:37.260384-04:00 wxbox weectl[7147]: INFO weewx.manager: >> Created and initialized table 'archive' in database 'weewx_new' >> >> Am I just misunderstanding what this tool does? If so, is there a way to >> migrate an existing database to a new schema? >> >> On Saturday, June 27, 2026 at 9:54:37 PM UTC-4 John Smith wrote: >> >>> I've never heard of a V5 problem that bad, but I suppose it's possible. >>> >>> >>> Considering they have a smaller number of columns, it could be a case of >>> taking a long time to calculate the missing ones. >>> >>> On Sun, 28 Jun 2026 at 06:53, Tom Keffer <[email protected]> wrote: >>> >>>> This is smelling like a V5 performance issue. See the Wiki article >>>> *Troubleshooting >>>> V5.x performance >>>> <https://github.com/weewx/weewx/wiki/v5-performance-troubleshooting>. >>>> *What >>>> gives me pause is that you said that sometimes the reports don't finish >>>> even after several days of running. I've never heard of a V5 problem that >>>> bad, but I suppose it's possible. >>>> >>>> Try putting appTemp in the database and see what happens. >>>> >>>> -tk >>>> >>>> >>>> >>>> >>>> On Sat, Jun 27, 2026 at 1:41 PM [email protected] <[email protected]> >>>> wrote: >>>> >>>>> >>>>> > Struggles to create the charts ? Lets see some logs showing that. >>>>> You did not post any debug=1 logs. >>>>> >>>>> I did, in fact, 400+ lines' worth. Nothing that I saw there about >>>>> image generation, but it's certainly possible I overlooked it. I >>>>> inferred >>>>> the "struggle" from the fact that, at that time, index.html had updated >>>>> but >>>>> none of the charts had. Over time (30+ minutes), the charts gradually >>>>> updated. >>>>> >>>>> > The only thing I can think of along those lines is weewx being slow >>>>> calculating things you refer to in your skin that are not elements the db >>>>> but that shouldn’t happen with a clean v5 installation tunning unaltered >>>>> Seasons >>>>> >>>>> If it's still using the existing database, which it was, that seems >>>>> plausible. Rich also mentioned something similar. >>>>> >>>>> > I’m thinking you have the old wview not wview-extended db perhaps. >>>>> You should have around 113 elements in the archive table not the small >>>>> old >>>>> schema with 52 or so. >>>>> >>>>> I think you're right. `describe archive` lists 53 rows. Looks like >>>>> some reading at >>>>> https://weewx.com/docs/5.1/custom/database/#modify-the-schema-of-an-existing-database >>>>> >>>>> is in order. >>>>> >>>>> On Saturday, June 27, 2026 at 4:31:07 PM UTC-4 Vince Skahan wrote: >>>>> >>>>>> Struggles to create the charts ? Lets see some logs showing that. You >>>>>> did not post any debug=1 logs. >>>>>> >>>>>> The only thing I can think of along those lines is weewx being slow >>>>>> calculating things you refer to in your skin that are not elements the >>>>>> db >>>>>> but that shouldn’t happen with a clean v5 installation tunning unaltered >>>>>> Seasons. >>>>>> >>>>>> On Saturday, June 27, 2026 at 12:42:32 PM UTC-7 Tom Keffer wrote: >>>>>> >>>>>>> On Sat, Jun 27, 2026 at 12:03 PM [email protected] <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> > MySQL server. You had a "Cannot connect" error on startup, so >>>>>>>> there is something flaky about the connection. >>>>>>>> >>>>>>>> It seems weewx starts up earlier than mysql, and thus fails to >>>>>>>> connect. It also seems to shut down earlier (on system shutdown or >>>>>>>> reboot) >>>>>>>> than mysql, leading to errors then. >>>>>>>> >>>>>>>> Enabling the general query log and following it doesn't show >>>>>>>> anything obvious. Nothing gets logged to the slow query log. >>>>>>>> >>>>>>> >>>>>>> It should show the types of queries running and how long they take. >>>>>>> Are you saying they were all quick? The time *must* be going >>>>>>> somewhere! >>>>>>> >>>>>>> Trying to rule out sources of a slowdown, I moved the VM storage to >>>>>>>> a local disk (it had been over the network), and again rebooted. And >>>>>>>> it >>>>>>>> now quickly generates the index.html page for the Seasons report, but >>>>>>>> it >>>>>>>> still seems to be struggling to create the -- >>>>>>>> >>>>>>> >>>>>>> Good idea. Try using sqlite and see if that's faster. >>>>>>> >>>>>>> -tk >>>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion visit >>>>> https://groups.google.com/d/msgid/weewx-user/367d2d60-042b-4ac0-a5e2-b836b7d0e6b0n%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/weewx-user/367d2d60-042b-4ac0-a5e2-b836b7d0e6b0n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "weewx-user" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>> To view this discussion visit >>>> https://groups.google.com/d/msgid/weewx-user/CAPq0zEC1O3r3B_ckvhXr%2Ba%3Dm3YY1gKKxJQFrhs9DtLs6j6GmNw%40mail.gmail.com >>>> >>>> <https://groups.google.com/d/msgid/weewx-user/CAPq0zEC1O3r3B_ckvhXr%2Ba%3Dm3YY1gKKxJQFrhs9DtLs6j6GmNw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> > To view this discussion visit >> https://groups.google.com/d/msgid/weewx-user/f352da7b-efd3-4e9d-a98e-a01c5dd5c88bn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/weewx-user/f352da7b-efd3-4e9d-a98e-a01c5dd5c88bn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/weewx-user/fad514fd-0b39-441d-9e6d-3bc962945cb0n%40googlegroups.com.
