I am having the same problem. I have about 130,000 different datetimes (records) in weewx.sdb. I also tried moving it to a ramdisk. Still extremely slow. Any help please? Should I run calc-missing on individual timeslices like one week periods? The data covers one year of measurements.
On Monday, March 14, 2022 at 7:50:36 PM UTC-7 graha...@gmail.com wrote: > i reduced the --calc-missing time from 7 *hours* to 7 *minutes* by such a > simple trick that i kick myself for not seeing it earlier - i moved the > database to ramdisk and symlinked it under archive, ran wee_database, then > moved the database back. chalk this one under 'handy tips' > > On Thursday, 10 March 2022 at 11:09:25 am UTC+11 graha...@gmail.com wrote: > >> for gw1000 dataset, i had been using top and iotop previously to >> ascertain that cpu, memory and i/o usage were extremely low, and ps-efl >> showed it was spending its time waiting on interrupt. i would usually just >> conclude it was slow disk and spending all its time waiting on i/o >> completion *except* this is only for the smaller gw1000 dataset not the >> larger vp2 dataset. it is something to do with the different nature of the >> data (perhaps something as simple as different missing data being calc’ed) >> >> to give an idea of the magnitude of the difference, using built-in shell >> time to run command: >> >> >> *recs* *real /sec* *user /sec* *sys /sec* *Idle /%* >> vp2 --rebuild-daily 505,336 165 148 2 9 >> vp2 --calc-missing 505,336 571 525 18 5 >> gw1000 --rebuild-daily 162,882 86 81 1 5 >> gw1000 --calc-missing 162,882 23,758 301 13 99 >> > [snip] > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/714d7461-fb59-4b36-9350-cff877203516n%40googlegroups.com.
