This could be a permissions problem on /var/lib/weewx/weewx.sdb, from memory a default install does not give an unprivileged user write permission to weewx.sdb. Try prefixing your wee_database commands (ie --drop-daily and --rebuild-daily) with sudo.
Gary On Thursday, 2 August 2018 04:40:37 UTC+10, Paul_N wrote: > > Hi Andrew, I finally managed to find some spare time to try out your > latest suggestion, sorry for the delay. Here's the result of running those > two commands (please note that I used "sudo find" else I get lots of > "permission denied" messages from numerous non-related RPi directories). > Does this shed any new light on things ? > Thanks again, Paul > > pi@raspberrypi:~ $ sudo find / -name weewx.conf > /etc/weewx/logwatch/conf/services/weewx.conf > /etc/weewx/logwatch/conf/logfiles/weewx.conf > /etc/weewx/rsyslog.d/weewx.conf > /etc/weewx/apache/conf.d/weewx.conf > /etc/weewx/weewx.conf > find: ‘/run/user/1000/gvfs’: Permission denied > > pi@raspberrypi:~ $ sudo find / -name weewx.sdb > /var/lib/weewx/weewx.sdb > find: ‘/run/user/1000/gvfs’: Permission denied > pi@raspberrypi:~ $ > > ==================================================================== > > On Wednesday, August 1, 2018 at 12:08:31 PM UTC+1, Paul_N wrote: >> >> Thanks for that suggestion Andrew, sounds very possible ! I'll check it >> out later today when I next get a spare moment and will let you know how I >> get on. >> Thanks again, Paul >> >> On Wednesday, August 1, 2018 at 11:57:36 AM UTC+1, Andrew Milner wrote: >>> >>> sorry - last one should have been >>> find / -name weewx.sdb >>> >>> >>> >>> On Wednesday, 1 August 2018 13:56:46 UTC+3, Andrew Milner wrote: >>>> >>>> My suspicion is that you have either got two weewx.conf files or two >>>> .sdb files and SQLite is looking in one place whilst drop daly is dropping >>>> tables from a different .sdb file. >>>> >>>> You can check by using find >>>> find / -name weewx.conf >>>> and >>>> find / -name archive.sdb >>>> >>>> >>>> >>>> On Wednesday, 1 August 2018 13:26:04 UTC+3, Paul_N wrote: >>>>> >>>>> Hi Andrew, thanks for this - and I think we're getting somewhere : it >>>>> would appear that the drop-daily command is not working. I followed your >>>>> first four steps and attach below the output from my terminal. As you'll >>>>> see the initial query found the rogue outTemp min values but they were >>>>> still there when I repeated that query AFTER doing the drop-daily >>>>> command. >>>>> So, if drop-daily is supposed to clear these values then it has not >>>>> worked. >>>>> >>>>> Any suggestions please ? >>>>> Thanks again, Paul (ps please note that all the temperature values >>>>> below are actually in Fahrenheit because that is how my db is set up, >>>>> hence >>>>> the rogue min values are actually below 5 deg F). >>>>> >>>>> [ ok ] Stopping weewx (via systemctl): weewx.service. >>>>> weewx stopped >>>>> >>>>> pi@raspberrypi:/var/lib/weewx $ sqlite3 weewx.sdb >>>>> SQLite version 3.16.2 2017-01-06 16:32:41 >>>>> Enter ".help" for usage hints. >>>>> sqlite> select dateTime, min from archive_day_outTemp where min < 5; >>>>> 1523919600|-3.46 >>>>> 1524006000|-3.64 >>>>> 1524438000|-3.82 >>>>> sqlite> >>>>> >>>>> pi@raspberrypi:/var/lib/weewx $ cd /etc/weewx >>>>> pi@raspberrypi:/etc/weewx $ wee_database weewx.conf --drop-daily >>>>> Using configuration file weewx.conf >>>>> Using database binding 'wx_binding', which is bound to database >>>>> 'archive_sqlite' >>>>> Proceeding will delete all your daily summaries from database >>>>> 'weewx.sdb' >>>>> Are you sure you want to proceed (y/n)? y >>>>> Dropping daily summary tables from 'weewx.sdb' ... >>>>> Daily summary tables dropped from database 'weewx.sdb' in 0.04 seconds >>>>> >>>>> pi@raspberrypi:/etc/weewx $ cd /var/lib/weewx >>>>> pi@raspberrypi:/var/lib/weewx $ sqlite3 weewx.sdb >>>>> SQLite version 3.16.2 2017-01-06 16:32:41 >>>>> Enter ".help" for usage hints. >>>>> sqlite> select dateTime, min from archive_day_outTemp where min < 5; >>>>> 1523919600|-3.46 >>>>> 1524006000|-3.64 >>>>> 1524438000|-3.82 >>>>> sqlite> >>>>> pi@raspberrypi:/var/lib/weewx $ >>>>> >>>>> =============================================================== >>>>> >>>>> On Wednesday, August 1, 2018 at 11:04:08 AM UTC+1, Andrew Milner wrote: >>>>>> >>>>>> 1. stop weewx >>>>>> >>>>>> 2. try and see if your rogue data is in the summaries: >>>>>> >>>>>> select dateTime, min from archive_day_outTemp where min < -19; >>>>>> >>>>>> hopefully you will find the rogue day!! >>>>>> >>>>>> >>>>>> 3. try and drop daily again >>>>>> >>>>>> 4. check that they have dropped by repeating the command: >>>>>> select dateTime,min from archive_day_outTemp where min < -19; >>>>>> >>>>>> Hopefully none will be found - in fact you may even get an error. >>>>>> >>>>>> 5. check there is no rogue value in the archive with >>>>>> select * from archive where outTemp < -19; >>>>>> >>>>>> 6. rebuild daily - this will rebuild all the archive_day tables from >>>>>> the archive data >>>>>> >>>>>> 7. check the summary table no longer contains the rogue data >>>>>> select * from archive_day_outTemp where min < -19; >>>>>> >>>>>> Hopefully you will have solved it. >>>>>> >>>>>> How long rebuild takes depends on loads of things, but should take a >>>>>> few seconds at least I would have thought and not been instantaneous. >>>>>> >>>>>> DO REMEMBER THE ; ON THE END OF SQL COMMANDS >>>>>> >>>>>> >>>>>> On Wednesday, 1 August 2018 11:13:01 UTC+3, Paul_N wrote: >>>>>>> >>>>>>> Hi Andrew, thanks for that suggestion. I have checked the database >>>>>>> with that query and can confirm that there are no rogue values left in >>>>>>> there. So the problem appears to be in the re-calculation of the >>>>>>> minimum >>>>>>> outdoor temperature for that month. I do not really understand exactly >>>>>>> what >>>>>>> the "rebuild-daily" command does - is that when these values get >>>>>>> calculated >>>>>>> ? If so, then maybe my observation that this command completed >>>>>>> instantly >>>>>>> might be relevant. There is a about 8 months of data in the database so >>>>>>> should it have taken some while to run, as suggested in Tom K's wiki ? >>>>>>> >>>>>>> Thanks again for any advice you could give me, >>>>>>> Regards, Paul >>>>>>> >>>>>>> On Wednesday, August 1, 2018 at 2:55:37 AM UTC+1, Andrew Milner >>>>>>> wrote: >>>>>>>> >>>>>>>> have you rechecked the database with >>>>>>>> >>>>>>>> select dateTime, outTemp from archive where outTemp < 19; >>>>>>>> >>>>>>>> to see if there are still any rogue temperature records in there? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tuesday, 31 July 2018 21:24:15 UTC+3, Paul_N wrote: >>>>>>>>> >>>>>>>>> Hi, I'd appreciate some advice on the problem I've been having >>>>>>>>> with removing some erroneous outside temperature readings from my >>>>>>>>> weewx >>>>>>>>> system which occurred very early on in my set-up when the sensor was >>>>>>>>> not >>>>>>>>> connected - and before I discovered the value of setting StdQC >>>>>>>>> properly ! >>>>>>>>> The erroneous values were very low outside temperatures (-19.9 deg >>>>>>>>> C), >>>>>>>>> causing the minimum values to be reported for that month and for the >>>>>>>>> year >>>>>>>>> to be incorrect. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> I followed the steps described by Tom K in his wiki guide >>>>>>>>> <https://github.com/weewx/weewx/wiki/Cleaning-up-old-'bad'-data> >>>>>>>>> and all seemed to go well: I was able to locate and remove (set to >>>>>>>>> null) >>>>>>>>> the erroneous outTemp readings in the database and then drop and >>>>>>>>> rebuild >>>>>>>>> the daily summaries OK. As advised elsewhere on this forum I also >>>>>>>>> deleted >>>>>>>>> all the PNG,HTML and NOAA text files so they were re-created after I >>>>>>>>> did >>>>>>>>> the database editing. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> The good news is that the png chart which shows monthly Min/Max >>>>>>>>> outside temperatures no longer shows these erroneous values and so I >>>>>>>>> am >>>>>>>>> encouraged to think that they really are gone from the database. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> However, the NOAA summary for that month (and also the yearly >>>>>>>>> one) still shows this bad data so it seems that whatever calculates >>>>>>>>> the >>>>>>>>> minimum outside temperature has not recalculated it for some reason. >>>>>>>>> Please >>>>>>>>> note that these NOAA summaries have definitely been recreated since >>>>>>>>> the >>>>>>>>> database editing as I deleted them after doing it and before >>>>>>>>> re-starting >>>>>>>>> weewx. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> One thing I did note which could be indicative is that although >>>>>>>>> the "rebuild-daily" command appeared to execute OK, it completed >>>>>>>>> almost >>>>>>>>> instantaneously whereas Tom's wiki guide led me to believe it would >>>>>>>>> take >>>>>>>>> some while. Could it be that this has not actually worked and hence >>>>>>>>> is the >>>>>>>>> cause of my problem ? How can I check to see if it has done what is >>>>>>>>> expected ? >>>>>>>>> >>>>>>>>> >>>>>>>>> Just in case it's relevant, here is what I saw on the terminal >>>>>>>>> when dropping and rebuilding the daily summaries: >>>>>>>>> >>>>>>>>> >>>>>>>>> pi@raspberrypi:/etc/weewx $ wee_database weewx.conf --drop-daily >>>>>>>>> >>>>>>>>> Using configuration file weewx.conf >>>>>>>>> >>>>>>>>> Using database binding 'wx_binding', which is bound to database >>>>>>>>> 'archive_sqlite' >>>>>>>>> >>>>>>>>> Proceeding will delete all your daily summaries from database >>>>>>>>> 'weewx.sdb' >>>>>>>>> >>>>>>>>> Are you sure you want to proceed (y/n)? y >>>>>>>>> >>>>>>>>> Dropping daily summary tables from 'weewx.sdb' ... >>>>>>>>> >>>>>>>>> Daily summary tables dropped from database 'weewx.sdb' in 0.04 >>>>>>>>> seconds >>>>>>>>> >>>>>>>>> >>>>>>>>> pi@raspberrypi:/etc/weewx $ wee_database weewx.conf --rebuild-daily >>>>>>>>> >>>>>>>>> Using configuration file weewx.conf >>>>>>>>> >>>>>>>>> Using database binding 'wx_binding', which is bound to database >>>>>>>>> 'archive_sqlite' >>>>>>>>> >>>>>>>>> All daily summaries will be rebuilt. >>>>>>>>> >>>>>>>>> Proceed (y/n)? y >>>>>>>>> >>>>>>>>> Rebuilding daily summaries in database 'weewx.sdb' ... >>>>>>>>> >>>>>>>>> Daily summaries up to date in 'weewx.sdb' >>>>>>>>> >>>>>>>>> >>>>>>>>> With thanks in advance for any help you could give to enable me to >>>>>>>>> track down this problem, >>>>>>>>> >>>>>>>>> Regards, Paul >>>>>>>>> >>>>>>>>> >>>>>>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
