i reduced the --calc-missing time from 7 *hours* to 7 *minutes* by such a simple trick that i kick myself for not seeing it earlier - i moved the database to ramdisk and symlinked it under archive, ran wee_database, then moved the database back. chalk this one under 'handy tips'
On Thursday, 10 March 2022 at 11:09:25 am UTC+11 [email protected] wrote: > for gw1000 dataset, i had been using top and iotop previously to ascertain > that cpu, memory and i/o usage were extremely low, and ps-efl showed it was > spending its time waiting on interrupt. i would usually just conclude it > was slow disk and spending all its time waiting on i/o completion *except* > this is only for the smaller gw1000 dataset not the larger vp2 dataset. it > is something to do with the different nature of the data (perhaps something > as simple as different missing data being calc’ed) > > to give an idea of the magnitude of the difference, using built-in shell > time to run command: > > > *recs* *real /sec* *user /sec* *sys /sec* *Idle /%* > vp2 --rebuild-daily 505,336 165 148 2 9 > vp2 --calc-missing 505,336 571 525 18 5 > gw1000 --rebuild-daily 162,882 86 81 1 5 > gw1000 --calc-missing 162,882 23,758 301 13 99 > [snip] -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/f339d818-08b1-4e8b-ae4b-3e9ea1eaf9cdn%40googlegroups.com.
