Re: [weewx-user] Re: Weewx generating reports two days head

[Mike Dame]

I finally was able to take a little time to dig through some of the
scripts, and after deleting pid file, and  issuing  a "systemctl
daemon-reload", and then weewx service finally restarted. It took a little
while to dump and record the missed records.

I think I am good now.

Thanks for everyone's assistance.

On Wed, Dec 27, 2017 at 7:15 AM, Mike Dame  wrote:

> Tried a delete from archive after midnight 12/25 that did not work after
> drop and rebuild still did not start. So per  request, I have attached the
> message logs from the 22nd till 26th.
>
> I tried turning on debug in the config file, still did not log anything or
> start
> Anyway to start in foreground to see if there is any clue why it is not
> starting up
>
> On Tue, Dec 26, 2017 at 10:54 PM, gjr80  wrote:
>
>> I'm not necessarily seeing anything wrong here. I would take it from the
>> top again and this time post the entire log, covering the wee_database
>> operations and weeWX startup and any errors. Better to post an full and
>> complete log rather than disjointed excerpts, far easier to read what is
>> happening. systemctl status is great for looking at the state of the weeWX
>> daemon but not much chop for looking at logs, it is like reading them
>> through a drinking straw.
>>
>> Gary
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "weewx-user" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>> pic/weewx-user/RezDiRbbCks/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> weewx-user+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Re: Weewx generating reports two days head

[Mike Dame]

So it will not restart, I copied back the backup database. I tried with and
without the sql drop, now the database will not start... :(

pi@raspberrypi:~ $ sudo cp -p /var/tmp/weewx.sdb.backup
/var/lib/weewx/weewx.sdb

pi@raspberrypi:~ $ sudo sqlite3 /var/lib/weewx/weewx.sdb
SQLite version 3.8.7.1 2014-10-29 13:59:56
Enter ".help" for usage hints.
sqlite>
sqlite> DELETE FROM archive WHERE dateTime > 1514181600;
sqlite> .quit

pi@raspberrypi:~ $ sudo wee_database --drop-daily
Using configuration file /etc/weewx/weewx.conf
Using database binding 'wx_binding', which is bound to database
'archive_sqlite'
Proceeding will delete all your daily summaries from database 'weewx.sdb'
Are you sure you want to proceed (y/n)? y
Dropping daily summary tables from 'weewx.sdb' ...
Daily summary tables dropped from database 'weewx.sdb' in 2.58 seconds

pi@raspberrypi:~ $ sudo wee_database --rebuild-daily
Using configuration file /etc/weewx/weewx.conf
Using database binding 'wx_binding', which is bound to database
'archive_sqlite'
All daily summaries will be rebuilt.
Proceed (y/n)? y
Rebuilding daily summaries in database 'weewx.sdb' ...
Records processed: 738000; Last date: 2017-12-21 11:30:00 CST (1513877400)
Processed 738994 records to rebuild 2585 day summaries in 698.76
seconds
Rebuild of daily summaries in database 'weewx.sdb' complete

pi@raspberrypi:~ $ sudo systemctl start weewx;tail -f /var/log/messages
Dec 26 21:04:06 raspberrypi wee_database[18640]: manager: Processed 738994
records to backfill 2585 day summaries in 698.48 seconds
Dec 26 21:04:06 raspberrypi wee_database[18640]: Rebuild of daily summaries
in database 'weewx.sdb' complete
Dec 26 21:13:36 raspberrypi wee_database[18824]: manager: Dropping daily
summary tables from 'weewx.sdb' ...
Dec 26 21:13:38 raspberrypi wee_database[18824]: manager: Dropped daily
summary tables from database 'weewx.sdb'
Dec 26 21:13:45 raspberrypi wee_database[18832]: All daily summaries will
be rebuilt.
Dec 26 21:13:47 raspberrypi wee_database[18832]: manager: Created daily
summary tables
Dec 26 21:13:47 raspberrypi wee_database[18832]: Rebuilding daily summaries
in database 'weewx.sdb' ...
Dec 26 21:13:47 raspberrypi wee_database[18832]: manager: Starting backfill
of daily summaries
Dec 26 21:25:21 raspberrypi wee_database[18832]: manager: Processed 739683
records to backfill 2588 day summaries in 694.00 seconds
Dec 26 21:25:22 raspberrypi wee_database[18832]: Rebuild of daily summaries
in database 'weewx.sdb' complete
^C
pi@raspberrypi:~ $ ps -ef | grep weewx
pi   18967  1332  0 21:26 pts/000:00:00 grep --color=auto weewx
pi@raspberrypi:~ $


pi@raspberrypi:~ $ sudo systemctl -l status weewx
● weewx.service - LSB: weewx weather system
   Loaded: loaded (/etc/init.d/weewx)
   Active: active (exited) since Tue 2017-12-26 09:11:01 CST; 11h ago
  Process: 12968 ExecStop=/etc/init.d/weewx stop (code=exited,
status=0/SUCCESS)
  Process: 13199 ExecStart=/etc/init.d/weewx start (code=exited,
status=0/SUCCESS)

Dec 26 09:11:02 raspberrypi weewx[13213]: File
"/usr/share/weewx/weewx/engine.py", line 604, in setup_database
Dec 26 09:11:02 raspberrypi weewx[13213]:   _nrecs, _ndays =
dbmanager.backfill_day_summary() # @UnusedVariable
Dec 26 09:11:02 raspberrypi weewx[13213]: File
"/usr/share/weewx/weewx/manager.py", line 1428, in backfill_day_summary
Dec 26 09:11:02 raspberrypi weewx[13213]: 
timestamp_to_string(lastRecord)))
Dec 26 09:11:02 raspberrypi weewx[13213]:   ViolatedPrecondition:
lastUpdate(2017-12-26 08:55:00 CST (1514300100)) > lastRecord(2017-12-25
00:00:00 CST (1514181600))
Dec 26 09:11:02 raspberrypi weewx[13213]:   Exiting.
Dec 26 20:44:30 raspberrypi systemd[1]: Started LSB: weewx weather system.
Dec 26 20:45:47 raspberrypi systemd[1]: Started LSB: weewx weather system.
Dec 26 20:47:02 raspberrypi systemd[1]: Started LSB: weewx weather system.
Dec 26 21:05:26 raspberrypi systemd[1]: Started LSB: weewx weather system.
pi@raspberrypi:~ $



On Tue, Dec 26, 2017 at 1:49 PM, gjr80  wrote:

> Hi,
>
> Try dropping the daily summaries before rebuilding:
>
> sudo wee_database --drop-daily
> sudo wee_database --rebuild-daily
>
> Since 3.7.0 (I think) you can usually, but not always, rebuild without
> first dropping. I think this is one of those exceptions.
>
> Gary
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "weewx-user" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/weewx-user/RezDiRbbCks/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> weewx-user+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit 

[weewx-user] Re: Weewx generating reports two days head

[gjr80]

Hi,

Try dropping the daily summaries before rebuilding:

sudo wee_database --drop-daily
sudo wee_database --rebuild-daily

Since 3.7.0 (I think) you can usually, but not always, rebuild without first 
dropping. I think this is one of those exceptions.

Gary

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Weewx generating reports two days head

[Mike Dame]

Got an error with trying to rebuild database.

pi@raspberrypi:~ $ sudo wee_database --rebuild-daily --from 2017-12-25
Using configuration file /etc/weewx/weewx.conf
Using database binding 'wx_binding', which is bound to database 
'archive_sqlite'
Daily summaries from 2017-12-25 through the end will be rebuilt.
Proceed (y/n)? y
Rebuilding daily summaries in database 'weewx.sdb' ...
Traceback (most recent call last):
  File "/usr/bin/wee_database", line 793, in 
main()
  File "/usr/bin/wee_database", line 153, in main
rebuildDaily(config_dict, db_binding, options)
  File "/usr/bin/wee_database", line 262, in rebuildDaily
trans_days=20)
  File "/usr/share/weewx/weewx/manager.py", line 1428, in 
backfill_day_summary
timestamp_to_string(lastRecord)))
weewx.ViolatedPrecondition: lastUpdate(2017-12-26 08:55:00 CST 
(1514300100)) > lastRecord(2017-12-25 00:00:00 CST (1514181600))


On Monday, December 25, 2017 at 8:53:02 PM UTC-6, Mike Dame wrote:
>
> Last night I rebooted my raspberry running that is running "weewx 
> v3.8.0".  And the report are dated two days ahead, so today is the 25th of 
> Dec, reported that the temps are from 27th of Dec. I have double checked 
> the system clock is correct on the raspberry and the time on the "WMR200a" 
> is set correct, within minute of the raspberry.  notice the date stamp on 
> the database check below.
>
> The only error that I can see in the messages log is "wmr200: MainThread: 
> W Clock time unsynchronized", I did not really get any hits that would give 
> me a hint if this is the error or not.
>
> pi@raspberrypi:~ $ sudo  wee_database --check
> Using configuration file /etc/weewx/weewx.conf
> Using database binding 'wx_binding', which is bound to database 
> 'archive_sqlite'
> Checking daily summary tables version...
> Daily summary tables are at version 1.0
> Recommend running --update to recalculate interval weightings.
> Daily summary tables version check completed in 0.02 seconds.
> Preparing Null String Check, this may take awhile...
> *Checking record: 739537; Timestamp: 2017-12-27 04:31:37 CST (1514370697)*
> No null strings found.
> Completed Null String Check in 70.77 seconds.
> pi@raspberrypi:~ $ date
> *Mon 25 Dec 20:42:46 CST 2017*
> pi@raspberrypi:~ $ 
>
> Anybody got some ideas?
>
> Regards,
> Mike
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] Re: Weewx generating reports two days head

[Thomas Keffer]

Somehow the clock on your weather station or your RPi got a couple days
ahead of reality, perhaps when you rebooted. Do you have a real-time clock?
To figure out what happened you would have to go through the historical RPi
system logs from back when you rebooted.

In any case, no matter what the cause, the fix is the same: delete the "in
the future" database records and rebuild the daily summaries. Here's the
steps (NOT TESTED):

1. Stop weewx

2. Make a backup copy of your database.

sudo cp /var/lib/weewx/weewx.sdb /var/tmp/weewx.sdb.backup


3. Figure out what the last good record is in your database. Let's say it
was 0100 on 25-Dec-2017 CST (UTC-6). That would be 1514181600 unix epoch
time. This website  can help.

4. Make sure the tool sqlite3 is on your system

apt-get install sqlite3


5. Use it to delete the future records

sudo sqlite /var/lib/weewx/weewx.sdb
sqlite> DELETE FROM archive WHERE dateTime > 1514181600;
sqlite> .quit


6. Rebuild the daily summaries

sudo wee_database --rebuild-daily --from 2017-12-25


7. Restart weewx.

Let us know how it goes.

-tk


On Tue, Dec 26, 2017 at 1:01 PM, Mike Dame  wrote:

> I have attached two cycles, the log looks normal to me (but not an expert).
> Without doing a sql query to look at the database, it weekly temp graph
> looks like it is adding data on the correct date, but the current graph is
> plotting data that was dated wrong
>
> Mike
>
>
> On Monday, December 25, 2017 at 8:53:02 PM UTC-6, Mike Dame wrote:
>>
>> Last night I rebooted my raspberry running that is running "weewx
>> v3.8.0".  And the report are dated two days ahead, so today is the 25th of
>> Dec, reported that the temps are from 27th of Dec. I have double checked
>> the system clock is correct on the raspberry and the time on the "WMR200a"
>> is set correct, within minute of the raspberry.  notice the date stamp on
>> the database check below.
>>
>> The only error that I can see in the messages log is "wmr200: MainThread:
>> W Clock time unsynchronized", I did not really get any hits that would give
>> me a hint if this is the error or not.
>>
>> pi@raspberrypi:~ $ sudo  wee_database --check
>> Using configuration file /etc/weewx/weewx.conf
>> Using database binding 'wx_binding', which is bound to database
>> 'archive_sqlite'
>> Checking daily summary tables version...
>> Daily summary tables are at version 1.0
>> Recommend running --update to recalculate interval weightings.
>> Daily summary tables version check completed in 0.02 seconds.
>> Preparing Null String Check, this may take awhile...
>> *Checking record: 739537; Timestamp: 2017-12-27 04:31:37 CST (1514370697)*
>> No null strings found.
>> Completed Null String Check in 70.77 seconds.
>> pi@raspberrypi:~ $ date
>> *Mon 25 Dec 20:42:46 CST 2017*
>> pi@raspberrypi:~ $
>>
>> Anybody got some ideas?
>>
>> Regards,
>> Mike
>>
> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[weewx-user] Re: Weewx generating reports two days head

[Mike Dame]

I have attached two cycles, the log looks normal to me (but not an expert).
Without doing a sql query to look at the database, it weekly temp graph 
looks like it is adding data on the correct date, but the current graph is 
plotting data that was dated wrong

Mike

On Monday, December 25, 2017 at 8:53:02 PM UTC-6, Mike Dame wrote:
>
> Last night I rebooted my raspberry running that is running "weewx 
> v3.8.0".  And the report are dated two days ahead, so today is the 25th of 
> Dec, reported that the temps are from 27th of Dec. I have double checked 
> the system clock is correct on the raspberry and the time on the "WMR200a" 
> is set correct, within minute of the raspberry.  notice the date stamp on 
> the database check below.
>
> The only error that I can see in the messages log is "wmr200: MainThread: 
> W Clock time unsynchronized", I did not really get any hits that would give 
> me a hint if this is the error or not.
>
> pi@raspberrypi:~ $ sudo  wee_database --check
> Using configuration file /etc/weewx/weewx.conf
> Using database binding 'wx_binding', which is bound to database 
> 'archive_sqlite'
> Checking daily summary tables version...
> Daily summary tables are at version 1.0
> Recommend running --update to recalculate interval weightings.
> Daily summary tables version check completed in 0.02 seconds.
> Preparing Null String Check, this may take awhile...
> *Checking record: 739537; Timestamp: 2017-12-27 04:31:37 CST (1514370697)*
> No null strings found.
> Completed Null String Check in 70.77 seconds.
> pi@raspberrypi:~ $ date
> *Mon 25 Dec 20:42:46 CST 2017*
> pi@raspberrypi:~ $ 
>
> Anybody got some ideas?
>
> Regards,
> Mike
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Dec 26 06:36:48 raspberrypi weewx[786]: wmr200: Thread-6: I USB polling device 
thread exiting
Dec 26 06:36:48 raspberrypi weewx[786]: wmr200: MainThread: I USB polling 
thread expired
Dec 26 06:36:48 raspberrypi weewx[786]: wmr200: Thread-5: I Watchdog received 
shutdown
Dec 26 06:36:48 raspberrypi weewx[786]: wmr200: Thread-5: I Watchdog thread 
exiting
Dec 26 06:36:48 raspberrypi weewx[786]: wmr200: MainThread: I Watchdog thread 
expired
Dec 26 06:36:48 raspberrypi weewx[786]: wmr200: MainThread: I Received packet 
count live:17712 archive:12 control:1299
Dec 26 06:36:48 raspberrypi weewx[786]: wmr200: MainThread: I Received 
bytes:999864 sent bytes:14352
Dec 26 06:36:48 raspberrypi weewx[786]: wmr200: MainThread: I Packet archive 
queue len:3 live queue len:0
Dec 26 06:36:48 raspberrypi weewx[786]: wmr200: MainThread: W Exiting with 
packets still in archive queue cnt:3
Dec 26 06:36:48 raspberrypi weewx[786]: wmr200: MainThread: I Driver gracefully 
exiting
Dec 26 06:36:48 raspberrypi weewx[786]: engine: Terminating weewx version 3.8.0
Dec 26 06:37:11 raspberrypi pi: == Weewx Stopped 
=
Dec 26 06:37:29 raspberrypi weewx[11805]: engine: Initializing weewx version 
3.8.0
Dec 26 06:37:29 raspberrypi weewx[11805]: engine: Using Python 2.7.9 (default, 
Sep 17 2016, 20:26:04) #012[GCC 4.9.2]
Dec 26 06:37:29 raspberrypi weewx[11805]: engine: Platform 
Linux-4.9.35-v7+-armv7l-with-debian-8.0
Dec 26 06:37:29 raspberrypi weewx[11805]: engine: Locale is 'en_GB.UTF-8'
Dec 26 06:37:29 raspberrypi weewx[11805]: engine: pid file is /var/run/weewx.pid
Dec 26 06:37:29 raspberrypi weewx[11809]: engine: Using configuration file 
/etc/weewx/weewx.conf
Dec 26 06:37:29 raspberrypi weewx[11809]: engine: Loading station type WMR200 
(weewx.drivers.wmr200)
Dec 26 06:37:29 raspberrypi weewx[11809]: wmr200: MainThread: I driver version 
is 3.3.4
Dec 26 06:37:29 raspberrypi weewx[11809]: wmr200: MainThread: I sensor map is 
{'outTempBatteryStatus': 'battery_status_out', 'outHumidity': 'humidity_1', 
'rainRate': 'rain_rate', 'heatindex4': 'heatindex_5', 'rainTotal': 
'rain_total', 'rainBatteryStatus': 'battery_status_rain', 'heatindex6': 
'heatindex_7', 'extraTemp5': 'temperature_6', 'uvFault': 'uv_fault', 
'extraTemp2': 'temperature_3', 'heatindex': 'heatindex_1', 'extraTemp3': 
'temperature_4', 'rain24': 'rain_24', 'inHeatindex': 'heatindex_0', 'inTemp': 
'temperature_0', 'extraTemp1': 'temperature_2', 'hourRain': 'rain_hour', 
'windFault': 'wind_fault', 'windchill': 'windchill', 'clockUnsynchronized': 
'clock_unsynchronized', 'extraTemp7': 'temperature_8', 'rain': 'rain', 
'forecastIcon': 'forecast_icon', 'pressure': 'pressure', 'extraHumid4': 
'humidity_5', 'extraHumid5': 'humidity_6', 'extraHumid2': 'humidity_3', 
'extraHumid3': 'humidity_4', 'extraHumid1': 'humidity_2', 'extraTemp6': 
'temperature_7', 'heatindex5': 'heatindex_6', 'extraTemp4': 'temperature_5', 
'heatindex7': 'heatindex_8', 'altimeter': 'altimeter', 'heatindex1': 
'heatindex_2', 'heatindex2':