I meant 4.6.0.beta* So weewx has been running for a bit and that bad data is present.
leafTemp1 -26.0°F leafTemp2 -82.0°F leafWet1 64 leafWet2 8 soilTemp1 -26.0°F soilTemp2 -82.0°F soilTemp3 -58.0°F soilTemp4 -90.0°F soilMoist1 2 cb soilMoist2 8 cb soilMoist3 2 cb soilMoist4 0 cb On Wednesday, October 13, 2021 at 8:57:34 PM UTC-7 Bill Richter wrote: > I did almost all of what you suggested, but didn't run for 10 minutes. > Rather than try to remember , what I did exactly, I'll follow your very > detailed instructions. > > weewx# sqlite3 /var/db/weewx.sdb > SQLite version 3.27.2 2019-02-25 16:06:06 > Enter ".help" for usage hints. > > sqlite> UPDATE archive SET leafTemp1=NULL WHERE leafTemp1 IS NOT NULL; > sqlite> .q > weewx# > > weewx# wee_database --rebuild-daily > Using configuration file /etc/weewx/weewx.conf > Using database binding 'wx_binding', which is bound to database > 'archive_sqlite' > All daily summaries will be rebuilt. > Proceed (y/n)? y > Rebuilding daily summaries in database 'weewx.sdb' > > # This section can adjust data using calibration expressions. > > [StdCalibrate] > > [[Corrections]] > # For each type, an arbitrary calibration expression can be given. > # It should be in the units defined in the StdConvert section. > > > ############################################################################## > > weewx# date > Wed Oct 13 20:16:55 PDT 2021 > weewx# PYTHONPATH=/usr/share/weewx python3 -m weewx.drivers.vantage > >/tmp/vantage.dump > ^C > KeyboardInterrupt > weewx# date > Wed Oct 13 20:31:09 PDT 2021 > > weewx# grep leafTemp1 /tmp/vantage.dump > weewx# wc -l /tmp/vantage.dump > 424 /tmp/vantage.dump > weewx# > > weewx# sqlite3 /var/db/weewx.sdb > SQLite version 3.27.2 2019-02-25 16:06:06 > Enter ".help" for usage hints. > sqlite> select leafTemp1 from archive where leafTemp1 IS NOT NULL; > sqlite> .q > weewx# > > weewx# weewxd --config /etc/weewx/weewx.conf > /tmp/weewx.log > ^C > KeyboardInterrupt > > weewx# > weewx# date > Wed Oct 13 20:46:38 PDT 2021 > weewx# > > weewx# sqlite3 /var/db/weewx.sdb > SQLite version 3.27.2 2019-02-25 16:06:06 > Enter ".help" for usage hints. > sqlite> select leafTemp1 from archive where leafTemp1 IS NOT NULL; > -26.0 > -26.0 > -26.0 > -26.0 > -26.0 > -26.0 > -26.0 > sqlite> SELECT dateTime,datetime(dateTime, > 'unixepoch','localtime'),leafTemp1 FROM archive WHERE leafTemp1 IS NOT NULL; > 1634181300|2021-10-13 20:15:00|-26.0 > 1634181600|2021-10-13 20:20:00|-26.0 > 1634181900|2021-10-13 20:25:00|-26.0 > 1634182200|2021-10-13 20:30:00|-26.0 > 1634182500|2021-10-13 20:35:00|-26.0 > 1634182800|2021-10-13 20:40:00|-26.0 > 1634183100|2021-10-13 20:45:00|-26.0 > sqlite> > > weewx# grep leafTemp1 /tmp/vantage.dump > weewx# wc -l /tmp/vantage.dump > 424 /tmp/vantage.dump > weewx# > > weewx# wee_device --clear-memory > Using configuration file /etc/weewx/weewx.conf > Using Vantage driver version 3.2.3 (weewx.drivers.vantage) > Proceeding will erase all archive records. > Are you sure you wish to proceed (y/n)? y > Erasing all archive records ... > Archive records erased. > > This confirms my earlier testing, it's not coming from the console, > something is touching the LOOP packet, and it's the same value all the > time. This is not a 'new" issue with 4.5.0.beta*. > > SELECT dateTime,datetime(dateTime, 'unixepoch','localtime'),soilMoist4 > FROM archive WHERE soilMoist4 IS NOT 0.0; > > 1487341800|2017-02-17 06:30:00|8.0 > 1487342100|2017-02-17 06:35:00|8.0 > 1487342400|2017-02-17 06:40:00|8.0 > 1487342700|2017-02-17 06:45:00|8.0 > 1487343000|2017-02-17 06:50:00|8.0 > > > > On Wednesday, October 13, 2021 at 7:47:31 PM UTC-7 gjr80 wrote: > >> On Wednesday, 13 October 2021 at 17:37:33 UTC+10 [email protected] >> wrote: >> >>> DRIVER_NAME = 'Vantage' >>> DRIVER_VERSION = '3.2.3' >>> >> Good, current version of the driver. >> >> >>> output looks 'normal'. I let it run for a while, looking for leafTemp1, >>> nothing found, >>> >> So the output you refer - I am guessing that is from running the driver >> directly as I mentioned previously? How long did you let it run for, at >> least 10 minutes (twice your archive interval)? It only takes one loop >> packet during an archive interval to include leafTemp1 and leafTemp1 >> will appear in your database. >> >> >>> but the database is getting a new record every 5 minutes. >>> >> Hang on, were these records saved when running the driver directly? If so >> that shouldn't happen, running the driver directly just outputs loop >> packets to the console, nothing is sent to WeeWX and hence nothing written >> to database. Did you stop WeeWX before running the driver directly? >> >> >>> After setting to 0.0 and confirming no none 0.0 records. >>> >> Not sure what you mean by this. >> >> sqlite> select leafTemp1 from archive where leafTemp1 != 0.0; >>> -26.0 >>> -26.0 >>> -26.0 >>> sqlite> >>> >> When querying the database you might find it useful (and will help us) if >> you use a query similar to this: >> >> SELECT dateTime,datetime(dateTime, 'unixepoch','localtime'),leafTemp1 >> FROM archive WHERE leafTemp1 != 0.0; >> >> This will give something like the following: >> >> 1487015700|2017-02-14 05:55:00|2.0 >> >> where you can see the timestamp of the record. >> >> >>> {'dateTime': 1634110148, 'usUnits': 1, 'barometer': 30.104, 'inTemp': >>> 70.3, 'inHumidity': 31.0, 'outTemp': 42.2, 'windSpeed': 0.0, 'windSpeed10': >>> 0.0, 'windDir': 342.0, 'outHumidity': 47.0, 'rainRate': 0.0, 'stormRain': >>> 0.0, 'dayRain': 0.0, 'monthRain': 0.2, 'yearRain': 0.2, 'dayET': 0.0, >>> 'monthET': 0.0, 'yearET': 0.0, 'leafWet4': 0.0, 'insideAlarm': 0, >>> 'rainAlarm': 0, 'outsideAlarm1': 0, 'outsideAlarm2': 0, 'extraAlarm1': 0, >>> 'extraAlarm2': 0, 'extraAlarm3': 0, 'extraAlarm4': 0, 'extraAlarm5': 0, >>> 'extraAlarm6': 0, 'extraAlarm7': 0, 'extraAlarm8': 0, 'soilLeafAlarm1': 0, >>> 'soilLeafAlarm2': 0, 'soilLeafAlarm3': 0, 'soilLeafAlarm4': 0, >>> 'txBatteryStatus': 0, 'consBatteryVoltage': 4.7, 'forecastIcon': 6, >>> 'forecastRule': 44, 'sunrise': 1634134260, 'sunset': 1634174820, 'rain': >>> 0.0} >>> >> OK, this looks like a loop packet straight off the driver/station, all >> looks good. >> >> The plot thickens. >>> >> Yes. A little background on how your station works with WeeWX. The >> vantage driver obtains loop packets from your station every 2.5 odd >> seconds. Various WeeWX services process the incoming loop packets; notable >> here are StdCalibrate which applies corrections to the loop data and >> StdWXCalculate which adds derived obs to the loop packet (eg apparent >> temperature). WeeWX then accumulates the loop packet data and the process >> repeats. At the end of the archive interval (every five minutes in your >> case) WeeWX asks the driver/station for an archive record. In your case the >> station logger generates that archive record and sends it to the driver >> which sends it to WeeWX. Again those WeeWX services process the archive >> record. One important difference to how loop packets are processed is that >> WeeWX extracts whatever additional information it can from those >> accumulated loop packets and adds it to the archive record if it is not >> already in the archive record. WeeWX then saves this archive record to your >> database. This is a greatly simplified outline of what goes on and it can >> vary from user to user depending on settings. >> >> So what? There are a few mechanisms that come to mind that could see >> leafTemp1 data saved in your database. For that to happen leafTemp1 must >> appear in the archive record that WeeWX saves to database. The following >> possibilities come to mind: >> >> 1. leafTemp1 is appearing regularly or randomly in loop packets, WeeWX >> accumulates this data and when the archive record is being processed before >> saving to database WeeWX is extracting the accumulated leafTemp1 data >> and adding it to the archive record which is then saved to database. This >> appears to be unlikely given you did not see any leafTemp1 data in the >> loop packets when you ran the driver directly. Though this depends on how >> often the issue shows itself, if it's every record then close monitoring of >> the driver loop packets for one or two archive intervals should either >> prove or dispel this as the problem. If it's a 'once in a while' problem >> then this is harder to prove one way or another. >> >> 2. The logger in your console (that generates the archive record and >> sends it to the driver) is for some unknown reason adding leafTemp1 data >> to the archive record. WeeWX then processes the archive record (but doesn't >> add leafTemp1 - it's already there) and saves it to database. This is >> consistent with not seeing leafTemp1 in any loop packets. Possible I >> guess, seems an unusual failure though. Might benefit from clearing the >> logger using wee_device --clear-memory >> <http://weewx.com/docs/hardware.htm#vantage_clear_console_memory> to >> clear the logger memory. There is no way of accessing the archive record as >> emitted by the driver without patching the either the driver code or WeeWX >> code. >> >> 3. Some other service is setting leafTemp1. Given your config this is >> pretty unlikely plus there are no other non-core WeeWX services being run >> that could be misbehaving. >> >> What to do. First up I would set your leafTemp1 data in your database to >> None rather than 0.0. 0.0 is considered valid data, eg a temperature of >> 0 degrees is still valid. In WeeWX parlance, missing or no data is usually >> indicated by the python value None. To do this: >> >> 1. stop WeeWX >> >> 2. use an SQLite update query to set leafTemp1 to NULL: >> $ sqlite3 /var/lib/weewx/weewx.sdb >> sqlite> UPDATE archive SET leafTemp1=NULL WHERE leafTemp1 IS NOT NULL; >> sqlite> .q >> >> 3. rebuild the daily summary tables: >> $ wee_database --rebuild-daily >> >> Then I would remove the [StdCalibrate] [[Corrections]] setting you have >> for leafTemp1. >> >> Then run the driver directly for at least 10 minutes and verify that >> leafTemp1 does not appear in any loop packets. Post the output her if >> you wish. You can check the database if you like (but use LeafTemp1 IS >> NOT NULL rather than LeafTemp1 != 0.0 in your query) but there should be >> nothing in there. >> >> Now run WeeWX directly, this will display loop packets (lines starting >> with LOOP:) and archive records (lines starting with REC:) on the console. >> Let WeeWX run for at least 10 minutes and see it leafTemp1 appears >> anywhere. If it appears does it appear in LOOP: or REC: lines or both? Post >> the output here. Again you can check your database if you wish >> >> Gary >> >> >>> >>> On Tuesday, October 12, 2021 at 11:09:39 PM UTC-7 gjr80 wrote: >>> >>>> OK, nothing of consequence in weewx.conf. You can run the vantage >>>> driver directly and have it output loop packets direct to the console, >>>> this >>>> takes WeeWX out of the equation. You can run the vantage driver directly >>>> by: >>>> >>>> 1. stopping WeeWX >>>> 2. running the following command: >>>> >>>> $ PYTHONPATH=/usr/share/weewx python3 -m weewx.drivers.vantage >>>> >>>> You should see continuous loop packets on the console. Are any bogus >>>> fields showing? If not let it run for a while. You can post the output >>>> here. Control-C will shut that down. Would also be useful to know the >>>> driver version: >>>> >>>> $ PYTHONPATH=/usr/share/weewx python3 -m weewx.drivers.vantage —version >>>> >>>> Gary >>>> >>> -- You received this message because you are subscribed to the Google Groups "weewx-development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/7aabd73f-a1fc-4366-b7d0-699e0261a014n%40googlegroups.com.
