Well, looking a little further into the docs answers the first question: weectl device --set-interval=5 changes the interval on the device. It doesn't appear to change the database; archive still has ~5M rows.
On Monday, June 29, 2026 at 7:54:17 PM UTC-4 [email protected] wrote: > How would I do that with a Vantage Pro 2? And would that involve any > changes to the database? weewx.conf already includes archive_interval = > 300. > > On Monday, June 29, 2026 at 11:23:47 AM UTC-4 [email protected] wrote: > >> As Rich pointed out, your archive timing is set by hardware to 1 minute. >> If that is unintended, change it to the default 5 minute interval. >> >> On Sunday, June 28, 2026 at 4:45:38 PM UTC-4 [email protected] wrote: >> >>> > But why would the primary key specification have been dropped? >>> >>> I can't say for certain, but I think I can reconstruct what happened. I >>> mentioned earlier that I migrated the database from the remote host where >>> it lived to the system where I run weewx--I did a mysqldump there, then >>> mysql weewx < weewx.sql on the other end. When the latter finished, it gave >>> a "temporary file write failure." But since all the tables were there, and >>> archive seemed to have the right number of rows, I figured it wasn't a >>> problem. But I got the same error just now when running "ALTER TABLE >>> archive ADD PRIMARY KEY (dateTime);" which led to some digging, determining >>> that /tmp is a RAMdisk in Debian 13, adding RAM to the VM, rebooting, and >>> rerunning that command--which completed successfully. >>> >>> So it seems likely that the failure when I initially imported this >>> database was in setting the primary key. And now the database calc-missing >>> operation--which had stalled, not indicating any progress at all before >>> setting the primary key--is running quite speedily, though it still >>> (naturally) takes a while to get through 5M rows. Once that finished, >>> restarted weewx, and unsurprisingly Seasons completed much >>> faster--including generating all the NOAA data, about 8 minutes. Running >>> Me.teo from scratch, 12 minutes. A second run, 10 seconds. >>> >>> So while I think there's still some testing I'll need to do, I think >>> I'll call this solved. I don't think the primary key was the sole issue, >>> because I was having performance problems when the database was on the >>> remote host, and the key was present there. But it was definitely an issue >>> once I moved the database to the weewx host, and apparently the lack of the >>> appTemp column was the rest of it. >>> >>> On Sunday, June 28, 2026 at 2:56:09 PM UTC-4 Tom Keffer wrote: >>> >>>> Absolutely. It's critical that dateTime be set as the primary key. >>>> Under MySQL the primary key is not only indexed, but it sets how data is >>>> organized. Without it, any query would pretty much require scanning the >>>> whole database. >>>> >>>> But why would the primary key specification have been dropped? Do you >>>> have old backups to examine when it was dropped? >>>> >>>> -tk >>>> >>>> On Sun, Jun 28, 2026 at 11:44 AM [email protected] <[email protected]> >>>> wrote: >>>> >>>>> Got it, and I'm in the middle of running the calc-missing step right >>>>> now. But in reviewing the describe archive outputs from the old and >>>>> the new database, I see that dateTime is set as a primary key in the >>>>> new table and not in the old one. Am I wrong in thinking that's also >>>>> likely to impact performance. >>>>> >>>>> On Sunday, June 28, 2026 at 2:42:09 PM UTC-4 Tom Keffer wrote: >>>>> >>>>>> There is no reason to migrate to the new schema. It is not an >>>>>> "upgrade." It just offers more types. In your case, because you need >>>>>> only a >>>>>> single new type (appTemp) it would be much more efficient to just add it >>>>>> using the tool "weectl database add-column >>>>>> <https://www.weewx.com/docs/5.4/utilities/weectl-database/#add-a-new-observation-type-to-the-database> >>>>>> ": >>>>>> >>>>>> *weectl database add-column appTemp* >>>>>> >>>>>> >>>>>> However, while that will add a new column, it will not populate it >>>>>> with anything. To do that, use weectl database calc-missing >>>>>> <https://www.weewx.com/docs/5.4/utilities/weectl-database/#calculate-missing-derived-variables> >>>>>> : >>>>>> >>>>>> *weectl database calc-missing* >>>>>> >>>>>> >>>>>> I would advise doing a backup first. >>>>>> >>>>>> -tk >>>>>> >>>>>> On Sun, Jun 28, 2026 at 11:25 AM [email protected] <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hmmm. My MySQL database is using the old schema, and it seems >>>>>>> there's a good likelihood that's at least contributing to the >>>>>>> performance >>>>>>> problems I'm seeing. And while the link Tom gave describes adding a >>>>>>> specific column to the table, >>>>>>> https://weewx.com/docs/5.4/custom/database/#reconfigure-using-new-schema >>>>>>> >>>>>>> provides instructions that would seem to address migrating to the new >>>>>>> schema completely. So I tried them. Stopped weewx, made a backup of >>>>>>> the >>>>>>> mysql database using mysqldump, and edited weewx.conf to revert to >>>>>>> using >>>>>>> the mysql database, and to use the wview_extended schema. Then ran >>>>>>> weectl >>>>>>> database reconfigure--which died after just under a minute: >>>>>>> dan@wxbox ➜ ~ time sudo -u weewx weectl database reconfigure >>>>>>> Using configuration file /etc/weewx/weewx.conf >>>>>>> Copying database 'weewx' to 'weewx_new' >>>>>>> The new database will use the same unit system as the old ('US'). >>>>>>> Are you sure you wish to proceed (y/n)? y >>>>>>> [2] 7144 killed sudo -u weewx weectl database reconfigure >>>>>>> sudo -u weewx weectl database reconfigure 0.01s user 0.01s system >>>>>>> 0% cpu 53.676 total >>>>>>> >>>>>>> The new database is created, the new archive table is created in it, >>>>>>> and it appears to have all the necessary fields (describe archive >>>>>>> returns >>>>>>> 115 rows), but it's empty. The system log doesn't report any errors: >>>>>>> 2026-06-28T14:16:26.783311-04:00 wxbox weectl[7147]: INFO weectllib: >>>>>>> Initializing weectl version 5.4.0 >>>>>>> 2026-06-28T14:16:26.787432-04:00 wxbox weectl[7147]: INFO weectllib: >>>>>>> Command line: /usr/share/weewx/weectl.py database reconfigure >>>>>>> 2026-06-28T14:16:26.788797-04:00 wxbox weectl[7147]: INFO weectllib: >>>>>>> Using Python: 3.13.5 (main, May 5 2026, 21:05:52) [GCC 14.2.0] >>>>>>> 2026-06-28T14:16:26.788976-04:00 wxbox weectl[7147]: INFO weectllib: >>>>>>> Located at: /usr/bin/python3 >>>>>>> 2026-06-28T14:16:26.789025-04:00 wxbox weectl[7147]: INFO weectllib: >>>>>>> Platform: Linux-6.12.94+deb13-amd64-x86_64-with-glibc2.41 >>>>>>> 2026-06-28T14:16:26.789078-04:00 wxbox weectl[7147]: INFO weectllib: >>>>>>> Locale: 'en_US.UTF-8' >>>>>>> 2026-06-28T14:16:26.789130-04:00 wxbox weectl[7147]: INFO weectllib: >>>>>>> Entry path: /usr/share/weewx/weectl.py >>>>>>> 2026-06-28T14:16:26.789303-04:00 wxbox weectl[7147]: INFO weectllib: >>>>>>> WEEWX_ROOT: /etc/weewx >>>>>>> 2026-06-28T14:16:26.789712-04:00 wxbox weectl[7147]: INFO weectllib: >>>>>>> Config file: /etc/weewx/weewx.conf >>>>>>> 2026-06-28T14:16:26.789764-04:00 wxbox weectl[7147]: INFO weectllib: >>>>>>> User module: /etc/weewx/bin/user >>>>>>> 2026-06-28T14:16:26.789957-04:00 wxbox weectl[7147]: INFO weectllib: >>>>>>> Debug: 1 >>>>>>> 2026-06-28T14:16:26.790009-04:00 wxbox weectl[7147]: INFO weectllib: >>>>>>> User: weewx >>>>>>> 2026-06-28T14:16:26.790050-04:00 wxbox weectl[7147]: INFO weectllib: >>>>>>> Group: weewx >>>>>>> 2026-06-28T14:16:26.790186-04:00 wxbox weectl[7147]: INFO weectllib: >>>>>>> Groups: dialout weewx >>>>>>> 2026-06-28T14:16:37.260384-04:00 wxbox weectl[7147]: INFO >>>>>>> weewx.manager: Created and initialized table 'archive' in database >>>>>>> 'weewx_new' >>>>>>> >>>>>>> Am I just misunderstanding what this tool does? If so, is there a >>>>>>> way to migrate an existing database to a new schema? >>>>>>> >>>>>>> On Saturday, June 27, 2026 at 9:54:37 PM UTC-4 John Smith wrote: >>>>>>> >>>>>>>> I've never heard of a V5 problem that bad, but I suppose it's >>>>>>>>> possible. >>>>>>>> >>>>>>>> >>>>>>>> Considering they have a smaller number of columns, it could be a >>>>>>>> case of taking a long time to calculate the missing ones. >>>>>>>> >>>>>>>> On Sun, 28 Jun 2026 at 06:53, Tom Keffer <[email protected]> wrote: >>>>>>>> >>>>>>>>> This is smelling like a V5 performance issue. See the Wiki article >>>>>>>>> *Troubleshooting >>>>>>>>> V5.x performance >>>>>>>>> <https://github.com/weewx/weewx/wiki/v5-performance-troubleshooting>. >>>>>>>>> *What >>>>>>>>> gives me pause is that you said that sometimes the reports don't >>>>>>>>> finish >>>>>>>>> even after several days of running. I've never heard of a V5 problem >>>>>>>>> that >>>>>>>>> bad, but I suppose it's possible. >>>>>>>>> >>>>>>>>> Try putting appTemp in the database and see what happens. >>>>>>>>> >>>>>>>>> -tk >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, Jun 27, 2026 at 1:41 PM [email protected] <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> > Struggles to create the charts ? Lets see some logs showing >>>>>>>>>> that. You did not post any debug=1 logs. >>>>>>>>>> >>>>>>>>>> I did, in fact, 400+ lines' worth. Nothing that I saw there >>>>>>>>>> about image generation, but it's certainly possible I overlooked it. >>>>>>>>>> I >>>>>>>>>> inferred the "struggle" from the fact that, at that time, index.html >>>>>>>>>> had >>>>>>>>>> updated but none of the charts had. Over time (30+ minutes), the >>>>>>>>>> charts >>>>>>>>>> gradually updated. >>>>>>>>>> >>>>>>>>>> > The only thing I can think of along those lines is weewx being >>>>>>>>>> slow calculating things you refer to in your skin that are not >>>>>>>>>> elements the >>>>>>>>>> db but that shouldn’t happen with a clean v5 installation tunning >>>>>>>>>> unaltered >>>>>>>>>> Seasons >>>>>>>>>> >>>>>>>>>> If it's still using the existing database, which it was, that >>>>>>>>>> seems plausible. Rich also mentioned something similar. >>>>>>>>>> >>>>>>>>>> > I’m thinking you have the old wview not wview-extended db >>>>>>>>>> perhaps. You should have around 113 elements in the archive table >>>>>>>>>> not the >>>>>>>>>> small old schema with 52 or so. >>>>>>>>>> >>>>>>>>>> I think you're right. `describe archive` lists 53 rows. Looks >>>>>>>>>> like some reading at >>>>>>>>>> https://weewx.com/docs/5.1/custom/database/#modify-the-schema-of-an-existing-database >>>>>>>>>> >>>>>>>>>> is in order. >>>>>>>>>> >>>>>>>>>> On Saturday, June 27, 2026 at 4:31:07 PM UTC-4 Vince Skahan wrote: >>>>>>>>>> >>>>>>>>>>> Struggles to create the charts ? Lets see some logs showing >>>>>>>>>>> that. You did not post any debug=1 logs. >>>>>>>>>>> >>>>>>>>>>> The only thing I can think of along those lines is weewx being >>>>>>>>>>> slow calculating things you refer to in your skin that are not >>>>>>>>>>> elements the >>>>>>>>>>> db but that shouldn’t happen with a clean v5 installation tunning >>>>>>>>>>> unaltered >>>>>>>>>>> Seasons. >>>>>>>>>>> >>>>>>>>>>> On Saturday, June 27, 2026 at 12:42:32 PM UTC-7 Tom Keffer wrote: >>>>>>>>>>> >>>>>>>>>>>> On Sat, Jun 27, 2026 at 12:03 PM [email protected] < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> > MySQL server. You had a "Cannot connect" error on startup, >>>>>>>>>>>>> so there is something flaky about the connection. >>>>>>>>>>>>> >>>>>>>>>>>>> It seems weewx starts up earlier than mysql, and thus fails to >>>>>>>>>>>>> connect. It also seems to shut down earlier (on system shutdown >>>>>>>>>>>>> or reboot) >>>>>>>>>>>>> than mysql, leading to errors then. >>>>>>>>>>>>> >>>>>>>>>>>>> Enabling the general query log and following it doesn't show >>>>>>>>>>>>> anything obvious. Nothing gets logged to the slow query log. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> It should show the types of queries running and how long they >>>>>>>>>>>> take. Are you saying they were all quick? The time *must* be >>>>>>>>>>>> going somewhere! >>>>>>>>>>>> >>>>>>>>>>>> Trying to rule out sources of a slowdown, I moved the VM >>>>>>>>>>>>> storage to a local disk (it had been over the network), and again >>>>>>>>>>>>> rebooted. And it now quickly generates the index.html page for >>>>>>>>>>>>> the Seasons >>>>>>>>>>>>> report, but it still seems to be struggling to create the -- >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Good idea. Try using sqlite and see if that's faster. >>>>>>>>>>>> >>>>>>>>>>>> -tk >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "weewx-user" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> To view this discussion visit >>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/367d2d60-042b-4ac0-a5e2-b836b7d0e6b0n%40googlegroups.com >>>>>>>>>> >>>>>>>>>> <https://groups.google.com/d/msgid/weewx-user/367d2d60-042b-4ac0-a5e2-b836b7d0e6b0n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "weewx-user" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> >>>>>>>> To view this discussion visit >>>>>>>>> https://groups.google.com/d/msgid/weewx-user/CAPq0zEC1O3r3B_ckvhXr%2Ba%3Dm3YY1gKKxJQFrhs9DtLs6j6GmNw%40mail.gmail.com >>>>>>>>> >>>>>>>>> <https://groups.google.com/d/msgid/weewx-user/CAPq0zEC1O3r3B_ckvhXr%2Ba%3Dm3YY1gKKxJQFrhs9DtLs6j6GmNw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "weewx-user" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> >>>>>> To view this discussion visit >>>>>>> https://groups.google.com/d/msgid/weewx-user/f352da7b-efd3-4e9d-a98e-a01c5dd5c88bn%40googlegroups.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/weewx-user/f352da7b-efd3-4e9d-a98e-a01c5dd5c88bn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> >>>> To view this discussion visit >>>>> https://groups.google.com/d/msgid/weewx-user/fad514fd-0b39-441d-9e6d-3bc962945cb0n%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/weewx-user/fad514fd-0b39-441d-9e6d-3bc962945cb0n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/weewx-user/98591d96-2949-4953-bfdb-5952d7951701n%40googlegroups.com.
