If I run " $alltime.humidex.max ($alltime.humidex.max.degree_F) $alltime.humidex.maxtime.format($ALLTIMEFMT) once a day it takes ~3 minutes. This is not a problem yet. However, if the database size increases to the point where the calculation time exceeds 5 minutes, will it skip the next interval, do it a minute or two "late," or cause things to blow up? This is more of a curiosity than anything else, but it seems to me someone will run $alltime.<some derived parameter> that will take more than the loop interval to complete.
- Paul VE1DX On Wednesday, November 29, 2023 at 12:43:18 PM UTC-4 Paul Dunphy wrote: > I asked about the time in the context of providing feedback for > development. If it is something on my end, then it is irrelevant in > terms of an issue with v5 . . . it's a new SD card, and the CPU is doing > nothing else, so I suspect it is a full database search. > > One table I generate uses $alltime.humidex.max > ($alltime.humidex.max.degree_F) > $alltime.humidex.maxtime.format($ALLTIMEFMT) . . . I assume that would > have to go through the entire database and calculate the value for each > record every time. > > Table "Lake Echo extremes since 2015" (lower right of the page) uses > this. Maybe I need to just calculate it once a day instead of each > interval. Either way, it's me and not related to the development of > v5. :-) > > https://ve1dx.net/v5/ > > - Paul VE1DX > > On 2023-11-29 12:12 p.m., Tom Keffer wrote: > > The templates are taking 3 minutes or more to run. That could be > > because they are very complex, they use tags that require whole > > database searches, your disk is on the verge of failure, or the CPUs > > are busy doing something else. > > > > On Wed, Nov 29, 2023 at 7:33 AM Paul Dunphy <[email protected]> wrote: > > > > Nov 29 11:13:33 jade python3[362677]: weewx[362677] INFO > > user.sftp: sftpgenerator: transferred 27 files in 5.47 seconds > > Nov 29 11:15:15 jade python3[362677]: weewx[362677] INFO > > weewx.manager: Added record 2023-11-29 11:15:16 AST (1701270916) > > to database 'mem.sdb' > > Nov 29 11:15:15 jade python3[362677]: weewx[362677] INFO > > weewx.manager: Added record 2023-11-29 11:15:16 AST (1701270916) > > to daily summary in 'mem.sdb' > > Nov 29 11:15:15 jade python3[362677]: weewx[362677] INFO > > weewx.manager: Added record 2023-11-29 11:15:00 AST (1701270900) > > to database 'weewx.sdb' > > Nov 29 11:15:15 jade python3[362677]: weewx[362677] INFO > > weewx.manager: Added record 2023-11-29 11:15:00 AST (1701270900) > > to daily summary in 'weewx.sdb' > > Nov 29 11:17:01 jade CRON[539036]: pam_unix(cron:session): session > > opened for user root(uid=0) by (uid=0) > > Nov 29 11:17:01 jade CRON[539037]: (root) CMD (cd / && run-parts > > --report /etc/cron.hourly) > > Nov 29 11:17:01 jade CRON[539036]: pam_unix(cron:session): session > > closed for user root > > Nov 29 11:18:27 jade python3[362677]: weewx[362677] INFO > > weewx.cheetahgenerator: Generated 13 files for report > > StandardReport in 191.18 seconds > > Nov 29 11:18:27 jade python3[362677]: weewx[362677] INFO > > weewx.imagegenerator: Generated 12 images for report > > StandardReport in 0.41 seconds > > Nov 29 11:18:27 jade python3[362677]: weewx[362677] INFO > > weewx.reportengine: Copied 0 files to > > /home/pdunphy/weewx-data/public_html > > Nov 29 11:18:27 jade python3[362677]: weewx[362677] INFO > > weewx.cheetahgenerator: Generated 1 files for report mem in 0.02 > > seconds > > Nov 29 11:18:27 jade python3[362677]: weewx[362677] INFO > > weewx.imagegenerator: Generated 1 images for report mem in 0.04 > > seconds > > Nov 29 11:18:27 jade python3[362677]: weewx[362677] INFO > > weewx.reportengine: Copied 0 files to > > /home/pdunphy/weewx-data/public_html/mem > > Nov 29 11:18:28 jade python3[362677]: weewx[362677] INFO > > paramiko.transport: Connected (version 2.0, client OpenSSH_8.9p1) > > Nov 29 11:18:28 jade python3[362677]: weewx[362677] INFO > > paramiko.transport: Authentication (publickey) successful! > > Nov 29 11:18:30 jade python3[362677]: weewx[362677] INFO > > paramiko.transport.sftp: [chan 0] Opened sftp connection (server > > version 3) > > Nov 29 11:18:34 jade python3[362677]: weewx[362677] INFO > > paramiko.transport.sftp: [chan 0] sftp session closed. > > Nov 29 11:18:34 jade python3[362677]: weewx[362677] INFO > > user.sftp: sftpgenerator: transferred 27 files in 6.95 seconds > > Nov 29 11:20:15 jade python3[362677]: weewx[362677] INFO > > weewx.manager: Added record 2023-11-29 11:20:16 AST (1701271216) > > to database 'mem.sdb' > > Nov 29 11:20:15 jade python3[362677]: weewx[362677] INFO > > weewx.manager: Added record 2023-11-29 11:20:16 AST (1701271216) > > to daily summary in 'mem.sdb' > > Nov 29 11:20:15 jade python3[362677]: weewx[362677] INFO > > weewx.manager: Added record 2023-11-29 11:20:00 AST (1701271200) > > to database 'weewx.sdb' > > Nov 29 11:20:15 jade python3[362677]: weewx[362677] INFO > > weewx.manager: Added record 2023-11-29 11:20:00 AST (1701271200) > > to daily summary in 'weewx.sdb' > > Nov 29 11:23:25 jade python3[362677]: weewx[362677] INFO > > weewx.cheetahgenerator: Generated 13 files for report > > StandardReport in 189.78 seconds > > Nov 29 11:23:25 jade python3[362677]: weewx[362677] INFO > > weewx.imagegenerator: Generated 12 images for report > > StandardReport in 0.43 seconds > > Nov 29 11:23:25 jade python3[362677]: weewx[362677] INFO > > weewx.reportengine: Copied 0 files to > > /home/pdunphy/weewx-data/public_html > > Nov 29 11:23:26 jade python3[362677]: weewx[362677] INFO > > weewx.cheetahgenerator: Generated 1 files for report mem in 0.02 > > seconds > > Nov 29 11:23:26 jade python3[362677]: weewx[362677] INFO > > weewx.imagegenerator: Generated 1 images for report mem in 0.03 > > seconds > > Nov 29 11:23:26 jade python3[362677]: weewx[362677] INFO > > weewx.reportengine: Copied 0 files to > > /home/pdunphy/weewx-data/public_html/mem > > Nov 29 11:23:26 jade python3[362677]: weewx[362677] INFO > > paramiko.transport: Connected (version 2.0, client OpenSSH_8.9p1) > > Nov 29 11:23:26 jade python3[362677]: weewx[362677] INFO > > paramiko.transport: Authentication (publickey) successful! > > Nov 29 11:23:28 jade python3[362677]: weewx[362677] INFO > > paramiko.transport.sftp: [chan 0] Opened sftp connection (server > > version 3) > > Nov 29 11:23:31 jade python3[362677]: weewx[362677] INFO > > paramiko.transport.sftp: [chan 0] sftp session closed. > > Nov 29 11:23:31 jade python3[362677]: weewx[362677] INFO > > user.sftp: sftpgenerator: transferred 27 files in 5.72 seconds > > Nov 29 11:25:15 jade python3[362677]: weewx[362677] INFO > > weewx.manager: Added record 2023-11-29 11:25:16 AST (1701271516) > > to database 'mem.sdb' > > Nov 29 11:25:15 jade python3[362677]: weewx[362677] INFO > > weewx.manager: Added record 2023-11-29 11:25:16 AST (1701271516) > > to daily summary in 'mem.sdb' > > Nov 29 11:25:15 jade python3[362677]: weewx[362677] INFO > > weewx.manager: Added record 2023-11-29 11:25:00 AST (1701271500) > > to database 'weewx.sdb' > > Nov 29 11:25:15 jade python3[362677]: weewx[362677] INFO > > weewx.manager: Added record 2023-11-29 11:25:00 AST (1701271500) > > to daily summary in 'weewx.sdb' > > Nov 29 11:28:25 jade python3[362677]: weewx[362677] INFO > > weewx.cheetahgenerator: Generated 13 files for report > > StandardReport in 189.62 seconds > > Nov 29 11:28:25 jade python3[362677]: weewx[362677] INFO > > weewx.imagegenerator: Generated 12 images for report > > StandardReport in 0.41 seconds > > Nov 29 11:28:25 jade python3[362677]: weewx[362677] INFO > > weewx.reportengine: Copied 0 files to > > /home/pdunphy/weewx-data/public_html > > Nov 29 11:28:25 jade python3[362677]: weewx[362677] INFO > > weewx.cheetahgenerator: Generated 1 files for report mem in 0.02 > > seconds > > Nov 29 11:28:25 jade python3[362677]: weewx[362677] INFO > > weewx.imagegenerator: Generated 1 images for report mem in 0.04 > > seconds > > Nov 29 11:28:25 jade python3[362677]: weewx[362677] INFO > > weewx.reportengine: Copied 0 files to > > /home/pdunphy/weewx-data/public_html/mem > > Nov 29 11:28:26 jade python3[362677]: weewx[362677] INFO > > paramiko.transport: Connected (version 2.0, client OpenSSH_8.9p1) > > Nov 29 11:28:26 jade python3[362677]: weewx[362677] INFO > > paramiko.transport: Authentication (publickey) successful! > > Nov 29 11:28:27 jade python3[362677]: weewx[362677] INFO > > paramiko.transport.sftp: [chan 0] Opened sftp connection (server > > version 3) > > Nov 29 11:28:31 jade python3[362677]: weewx[362677] INFO > > paramiko.transport.sftp: [chan 0] sftp session closed. > > Nov 29 11:28:31 jade python3[362677]: weewx[362677] INFO > > user.sftp: sftpgenerator: transferred 27 files in 5.43 seconds > > > > > > On Wednesday, November 29, 2023 at 11:29:18 AM UTC-4 Tom Keffer wrote: > > > > Impossible to say without seeing the log. > > > > On Wed, Nov 29, 2023 at 6:57 AM Paul Dunphy <[email protected]> > > wrote: > > > > I am testing v5.0.0b15 in a venv (installed on a Debian-12 > > based 4 GB Pi 4B using Vince's script.) I'm using the > > Simulator to add records to a copy of my database with > > about 860,000 records. It's working well, but I note the > > new record is added at XX:05, XX:10, etc, and then the CPU > > usage jumps up for ~3 minutes. Then it uploads the HTML > > code to my VPS, and the CPU drops to idle until the next > > interval ~2 minutes later. > > > > Am I correct that the system is sweeping through the > > database, calculating the derived parameters' min, max, > > etc.? With versions before v5, the upload occurred > > seconds after the new record was added. > > > > - Paul VE1DX > > > > -- > > You received this message because you are subscribed to > > the Google Groups "weewx-development" group. > > To unsubscribe from this group and stop receiving emails > > from it, send an email to [email protected]. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/weewx-development/706963a2-71bd-4577-8bda-5e7a1457fb75n%40googlegroups.com > > < > https://groups.google.com/d/msgid/weewx-development/706963a2-71bd-4577-8bda-5e7a1457fb75n%40googlegroups.com?utm_medium=email&utm_source=footer > >. > > > > -- > > You received this message because you are subscribed to the Google > > Groups "weewx-development" group. > > To unsubscribe from this group and stop receiving emails from it, > > send an email to [email protected]. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/weewx-development/d3c02ca2-de0d-4191-8060-c59e7c95793an%40googlegroups.com > > < > https://groups.google.com/d/msgid/weewx-development/d3c02ca2-de0d-4191-8060-c59e7c95793an%40googlegroups.com?utm_medium=email&utm_source=footer > >. > > > > -- You received this message because you are subscribed to the Google Groups "weewx-development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/bd7c4109-c52c-4ccd-bd86-a65ca9741544n%40googlegroups.com.
