> If your reporting cycle is finishing within the 60 seconds archive
interval
It isn't. For a while, it was struggling to complete them within a
5-minute interval, but I think I have it reasonably-well sorted at
this point.
Right now, I'm running two skins, Me.teo and new-belchertown. And
while it isn't especially surprising that generating reports would be
more demanding on system resources, it is surprising (to me, at least)
how much more demanding it is. I'd been running Weewx (admittedly,
without local MySQL) on this VM with 2 GB of RAM and 2 cores for
years, without any issues. Now that I'm building reports with these
skins locally, weewx would die (killed by oom-killer) with 12 GB RAM
allocated to the VM--timing would be apparently-random, but it
wouldn't stay up more than a day. Since increasing the RAM to 16 GB,
I haven't seen that happen over the last few days. But right now,
systemctl status weewxreports it's using 11.2G of RAM, and its peak
since last restart was 11.4 GB.
The other issue I was running into was a whole lot of disk
thrashing--this was resolved by giving MariaDB a larger RAM buffer.
I'd expect thinning the archive would improve performance here by
quite a bit, but I don't think I really want to do that if I can help
it--but it does seem the five-minute reporting cycle going forward is
going to be needed.
On Monday, June 29, 2026 at 9:14:16 PM UTC-4 Tom Keffer wrote:
No need to delete anything. WeeWX works just fine with mixed
archive intervals.
If your reporting cycle is finishing within the 60 seconds archive
interval and you don't mind the bigger database, there's no reason
to do anything at all. Just keep the interval at one minute.
On Mon, Jun 29, 2026 at 5:39 PM John Smith <[email protected]>
wrote:
You would need to delete the extra rows using a SQL query, to
find rows that aren't every 5 minutes you can do
SELECT * FROM archive WHERE dateTime % 300 != 0;
On Tue, 30 Jun 2026 at 10:09, [email protected]
<[email protected]> wrote:
Well, looking a little further into the docs answers the
first question: weectl device --set-interval=5 changes the
interval on the device. It doesn't appear to change the
database; archive still has ~5M rows.
On Monday, June 29, 2026 at 7:54:17 PM UTC-4
[email protected] wrote:
How would I do that with a Vantage Pro 2? And would
that involve any changes to the database? weewx.conf
already includes archive_interval = 300.
On Monday, June 29, 2026 at 11:23:47 AM UTC-4
[email protected] wrote:
As Rich pointed out, your archive timing is set by
hardware to 1 minute. If that is unintended,
change it to the default 5 minute interval.
On Sunday, June 28, 2026 at 4:45:38 PM UTC-4
[email protected] wrote:
> But why would the primary key specification
have been dropped?
I can't say for certain, but I think I can
reconstruct what happened. I mentioned earlier
that I migrated the database from the remote
host where it lived to the system where I run
weewx--I did a mysqldump there, then mysql
weewx < weewx.sql on the other end. When the
latter finished, it gave a "temporary file
write failure." But since all the tables were
there, and archive seemed to have the right
number of rows, I figured it wasn't a problem.
But I got the same error just now when running
"ALTER TABLE archive ADD PRIMARY KEY
(dateTime);" which led to some digging,
determining that /tmp is a RAMdisk in Debian
13, adding RAM to the VM, rebooting, and
rerunning that command--which completed
successfully.
So it seems likely that the failure when I
initially imported this database was in
setting the primary key. And now the database
calc-missing operation--which had stalled, not
indicating any progress at all before setting
the primary key--is running quite speedily,
though it still (naturally) takes a while to
get through 5M rows. Once that finished,
restarted weewx, and unsurprisingly Seasons
completed much faster--including generating
all the NOAA data, about 8 minutes. Running
Me.teo from scratch, 12 minutes. A second
run, 10 seconds.
So while I think there's still some testing
I'll need to do, I think I'll call this
solved. I don't think the primary key was the
sole issue, because I was having performance
problems when the database was on the remote
host, and the key was present there. But it
was definitely an issue once I moved the
database to the weewx host, and apparently the
lack of the appTemp column was the rest of it.
On Sunday, June 28, 2026 at 2:56:09 PM UTC-4
Tom Keffer wrote:
Absolutely. It's critical that dateTime be
set as the primary key. Under MySQL the
primary key is not only indexed, but it
sets how data is organized. Without it,
any query would pretty much require
scanning the whole database.
But why would the primary key
specification have been dropped? Do you
have old backups to examine when it was
dropped?
-tk
On Sun, Jun 28, 2026 at 11:44 AM
[email protected] <[email protected]> wrote:
Got it, and I'm in the middle of
running the calc-missing step right
now. But in reviewing the describe
archive outputs from the old and the
new database, I see that dateTime is
set as a primary key in the new table
and not in the old one. Am I wrong in
thinking that's also likely to impact
performance.
On Sunday, June 28, 2026 at 2:42:09 PM
UTC-4 Tom Keffer wrote:
There is no reason to migrate to
the new schema. It is not an
"upgrade." It just offers more
types. In your case, because you
need only a single new type
(appTemp) it would be much more
efficient to just add it using the
tool "weectl database add-column
<https://www.weewx.com/docs/5.4/utilities/weectl-database/#add-a-new-observation-type-to-the-database>":
*weectl database add-column
appTemp*
However, while that will add a new
column, it will not populate it
with anything. To do that, use
weectl database calc-missing
<https://www.weewx.com/docs/5.4/utilities/weectl-database/#calculate-missing-derived-variables>:
*weectl database calc-missing*
I would advise doing a backup first.
-tk
On Sun, Jun 28, 2026 at 11:25 AM
[email protected]
<[email protected]> wrote:
Hmmm. My MySQL database is
using the old schema, and it
seems there's a good
likelihood that's at least
contributing to the
performance problems I'm
seeing. And while the link
Tom gave describes adding a
specific column to the table,
https://weewx.com/docs/5.4/custom/database/#reconfigure-using-new-schema
provides instructions that
would seem to address
migrating to the new schema
completely. So I tried them.
Stopped weewx, made a backup
of the mysql database using
mysqldump, and edited
weewx.conf to revert to using
the mysql database, and to use
the wview_extended schema.
Then ran weectl database
reconfigure--which died after
just under a minute:
dan@wxbox ➜ ~ time sudo -u
weewx weectl database reconfigure
Using configuration file
/etc/weewx/weewx.conf
Copying database 'weewx' to
'weewx_new'
The new database will use the
same unit system as the old
('US').
Are you sure you wish to
proceed (y/n)? y
[2] 7144 killed sudo -u
weewx weectl database reconfigure
sudo -u weewx weectl database
reconfigure 0.01s user 0.01s
system 0% cpu 53.676 total
The new database is created,
the new archive table is
created in it, and it appears
to have all the necessary
fields (describe archive
returns 115 rows), but it's
empty. The system log doesn't
report any errors:
2026-06-28T14:16:26.783311-04:00
wxbox weectl[7147]: INFO
weectllib: Initializing weectl
version 5.4.0
2026-06-28T14:16:26.787432-04:00
wxbox weectl[7147]: INFO
weectllib: Command line:
/usr/share/weewx/weectl.py
database reconfigure
2026-06-28T14:16:26.788797-04:00
wxbox weectl[7147]: INFO
weectllib: Using Python:
3.13.5 (main, May 5 2026,
21:05:52) [GCC 14.2.0]
2026-06-28T14:16:26.788976-04:00
wxbox weectl[7147]: INFO
weectllib: Located at:
/usr/bin/python3
2026-06-28T14:16:26.789025-04:00
wxbox weectl[7147]: INFO
weectllib: Platform:
Linux-6.12.94+deb13-amd64-x86_64-with-glibc2.41
2026-06-28T14:16:26.789078-04:00
wxbox weectl[7147]: INFO
weectllib: Locale: 'en_US.UTF-8'
2026-06-28T14:16:26.789130-04:00
wxbox weectl[7147]: INFO
weectllib: Entry path:
/usr/share/weewx/weectl.py
2026-06-28T14:16:26.789303-04:00
wxbox weectl[7147]: INFO
weectllib: WEEWX_ROOT: /etc/weewx
2026-06-28T14:16:26.789712-04:00
wxbox weectl[7147]: INFO
weectllib: Config file:
/etc/weewx/weewx.conf
2026-06-28T14:16:26.789764-04:00
wxbox weectl[7147]: INFO
weectllib: User module:
/etc/weewx/bin/user
2026-06-28T14:16:26.789957-04:00
wxbox weectl[7147]: INFO
weectllib: Debug: 1
2026-06-28T14:16:26.790009-04:00
wxbox weectl[7147]: INFO
weectllib: User: weewx
2026-06-28T14:16:26.790050-04:00
wxbox weectl[7147]: INFO
weectllib: Group: weewx
2026-06-28T14:16:26.790186-04:00
wxbox weectl[7147]: INFO
weectllib: Groups:
dialout weewx
2026-06-28T14:16:37.260384-04:00
wxbox weectl[7147]: INFO
weewx.manager: Created and
initialized table 'archive' in
database 'weewx_new'
Am I just misunderstanding
what this tool does? If so,
is there a way to migrate an
existing database to a new schema?
On Saturday, June 27, 2026 at
9:54:37 PM UTC-4 John Smith wrote:
I've never heard of a
V5 problem that bad,
but I suppose it's
possible.
Considering they have a
smaller number of columns,
it could be a case of
taking a long time to
calculate the missing ones.
On Sun, 28 Jun 2026 at
06:53, Tom Keffer
<[email protected]> wrote:
This is smelling like
a V5 performance
issue. See the Wiki
article
/Troubleshooting V5.x
performance
<https://github.com/weewx/weewx/wiki/v5-performance-troubleshooting>.
/What gives me pause
is that you said that
sometimes the reports
don't finish even
after several days of
running. I've never
heard of a V5 problem
that bad, but I
suppose it's possible.
Try putting appTemp in
the database and see
what happens.
-tk
/
/
On Sat, Jun 27, 2026
at 1:41 PM
[email protected]
<[email protected]> wrote:
> Struggles to
create the charts
? Lets see some
logs showing that.
You did not post
any debug=1 logs.
I did, in fact,
400+ lines' worth.
Nothing that I saw
there about image
generation, but
it's certainly
possible I
overlooked it. I
inferred the
"struggle" from
the fact that, at
that time,
index.html had
updated but none
of the charts had.
Over time (30+
minutes), the
charts gradually
updated.
> The only thing I
can think of along
those lines is
weewx being slow
calculating things
you refer to in
your skin that are
not elements the
db but that
shouldn’t happen
with a clean v5
installation
tunning unaltered
Seasons
If it's still
using the existing
database, which it
was, that seems
plausible. Rich
also mentioned
something similar.
> I’m thinking you
have the old wview
not wview-extended
db perhaps. You
should have around
113 elements in
the archive table
not the small old
schema with 52 or so.
I think you're
right. `describe
archive` lists 53
rows. Looks like
some reading at
https://weewx.com/docs/5.1/custom/database/#modify-the-schema-of-an-existing-database
is in order.
On Saturday, June
27, 2026 at
4:31:07 PM UTC-4
Vince Skahan wrote:
Struggles to
create the
charts ? Lets
see some logs
showing that.
You did not
post any
debug=1 logs.
The only thing
I can think of
along those
lines is weewx
being slow
calculating
things you
refer to in
your skin that
are not
elements the
db but that
shouldn’t
happen with a
clean v5
installation
tunning
unaltered Seasons.
On Saturday,
June 27, 2026
at 12:42:32 PM
UTC-7 Tom
Keffer wrote:
On Sat,
Jun 27,
2026 at
12:03 PM
[email protected]
<[email protected]>
wrote:
>
MySQL
server.
You
had a
"Cannot
connect"
error
on
startup,
so
there
is
something
flaky
about
the
connection.
It
seems
weewx
starts
up
earlier
than
mysql,
and
thus
fails
to
connect.
It
also
seems
to
shut
down
earlier
(on
system
shutdown
or
reboot)
than
mysql,
leading
to
errors
then.
Enabling
the
general
query
log
and
following
it
doesn't
show
anything
obvious.
Nothing
gets
logged
to the
slow
query log.
It should
show the
types of
queries
running
and how
long they
take. Are
you saying
they were
all quick?
The time
*must* be
going
somewhere!
Trying
to
rule
out
sources
of a
slowdown,
I
moved
the VM
storage
to a
local
disk
(it
had
been
over
the
network),
and
again
rebooted.
And it
now
quickly
generates
the
index.html
page
for
the
Seasons
report,
but it
still
seems
to be
struggling
to
create
the --
Good idea.
Try using
sqlite and
see if
that's faster.
-tk
--
You received this
message because
you are subscribed
to the Google
Groups
"weewx-user" group.
To unsubscribe
from this group
and stop receiving
emails from it,
send an email to
[email protected].
To view this
discussion visit
https://groups.google.com/d/msgid/weewx-user/367d2d60-042b-4ac0-a5e2-b836b7d0e6b0n%40googlegroups.com
<https://groups.google.com/d/msgid/weewx-user/367d2d60-042b-4ac0-a5e2-b836b7d0e6b0n%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this
message because you
are subscribed to the
Google Groups
"weewx-user" group.
To unsubscribe from
this group and stop
receiving emails from
it, send an email to
[email protected].
To view this
discussion visit
https://groups.google.com/d/msgid/weewx-user/CAPq0zEC1O3r3B_ckvhXr%2Ba%3Dm3YY1gKKxJQFrhs9DtLs6j6GmNw%40mail.gmail.com
<https://groups.google.com/d/msgid/weewx-user/CAPq0zEC1O3r3B_ckvhXr%2Ba%3Dm3YY1gKKxJQFrhs9DtLs6j6GmNw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
--
You received this message
because you are subscribed to
the Google Groups "weewx-user"
group.
To unsubscribe from this group
and stop receiving emails from
it, send an email to
[email protected].
To view this discussion visit
https://groups.google.com/d/msgid/weewx-user/f352da7b-efd3-4e9d-a98e-a01c5dd5c88bn%40googlegroups.com
<https://groups.google.com/d/msgid/weewx-user/f352da7b-efd3-4e9d-a98e-a01c5dd5c88bn%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you
are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and
stop receiving emails from it, send an
email to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/weewx-user/fad514fd-0b39-441d-9e6d-3bc962945cb0n%40googlegroups.com
<https://groups.google.com/d/msgid/weewx-user/fad514fd-0b39-441d-9e6d-3bc962945cb0n%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to
the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails
from it, send an email to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/weewx-user/98591d96-2949-4953-bfdb-5952d7951701n%40googlegroups.com
<https://groups.google.com/d/msgid/weewx-user/98591d96-2949-4953-bfdb-5952d7951701n%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the
Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/weewx-user/CAGTinV4X6qZZvro6RkTmbAdyx_uMkEU722sfyX4nBfeQhGOzgg%40mail.gmail.com
<https://groups.google.com/d/msgid/weewx-user/CAGTinV4X6qZZvro6RkTmbAdyx_uMkEU722sfyX4nBfeQhGOzgg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google
Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/weewx-user/f892e7e6-16ce-4344-bc58-a50df3dd3a5fn%40googlegroups.com
<https://groups.google.com/d/msgid/weewx-user/f892e7e6-16ce-4344-bc58-a50df3dd3a5fn%40googlegroups.com?utm_medium=email&utm_source=footer>.
