The only thing I can think of is that your system is taking an unusually long time to open the file being parsed. Try this version of fileparse.py. It will log the time the file is opened and closed, giving us a clue.
-tk On Thu, Jul 11, 2019 at 2:33 PM 'Ron Walker' via weewx-user < [email protected]> wrote: > FileParse version is 0.6 > > No errors in the log (except those caused by removing a sensor) as seen > below. > > Jul 11 17:22:17 WeatherPi weewx[1501]: engine: Initializing weewx version > 3.9.1 > Jul 11 17:22:17 WeatherPi weewx[1501]: engine: Using Python 2.7.13 > (default, Sep 26 2018, 18:42:22) #012[GCC 6.3.0 20170516] > Jul 11 17:22:17 WeatherPi weewx[1501]: engine: Platform > Linux-4.19.42-v7+-armv7l-with-debian-9.9 > Jul 11 17:22:17 WeatherPi weewx[1501]: engine: Locale is 'en_US.UTF-8' > Jul 11 17:22:17 WeatherPi weewx[1501]: engine: Using configuration file > /etc/weewx/weewx.conf > Jul 11 17:22:17 WeatherPi weewx[1501]: engine: Loading station type > FileParse (user.fileparse) > Jul 11 17:22:17 WeatherPi weewx[1501]: fileparse: data file is > /var/tmp/data.csv > Jul 11 17:22:17 WeatherPi weewx[1501]: fileparse: polling interval is 10.0 > Jul 11 17:22:17 WeatherPi weewx[1501]: fileparse: label map is > {'pressure': 'pressure', 'outTemp': 'outTemp', 'inTemp': 'inTemp', > 'outHumidity': 'outHumidity', 'inHumidity': 'inHumidity', 'windSpeed': > 'windSpeed', 'windDir': 'windDir', 'windGust': 'windGust', 'windGustDir': > 'windGustDir', 'supplyVoltage': 'supplyVoltage', 'lightLevel': 'radiation', > 'rain': 'rain'} > Jul 11 17:22:17 WeatherPi weewx[1501]: engine: StdConvert target unit is > 0x1 > Jul 11 17:22:17 WeatherPi weewx[1501]: wxcalculate: The following values > will be calculated: barometer=prefer_hardware, windchill=prefer_hardware, > dewpoint=prefer_hardware, appTemp=prefer_hardware, rainRate=software, > windrun=prefer_hardware, heatindex=prefer_hardware, > maxSolarRad=prefer_hardware, humidex=prefer_hardware, > pressure=prefer_hardware, inDewpoint=prefer_hardware, ET=prefer_hardware, > altimeter=prefer_hardware, cloudbase=prefer_hardware > Jul 11 17:22:17 WeatherPi weewx[1501]: wxcalculate: The following > algorithms will be used for calculations: altimeter=aaNOAA, maxSolarRad=RS > Jul 11 17:22:17 WeatherPi weewx[1501]: engine: Archive will use data > binding wx_binding > Jul 11 17:22:17 WeatherPi weewx[1501]: engine: Record generation will be > attempted in 'hardware' > Jul 11 17:22:17 WeatherPi weewx[1501]: engine: Using archive interval of > 300 seconds (specified in weewx configuration) > Jul 11 17:22:17 WeatherPi weewx[1501]: engine: Using binding 'wx_binding' > to database 'weewx.sdb' > Jul 11 17:22:17 WeatherPi weewx[1501]: manager: Starting backfill of daily > summaries > Jul 11 17:22:17 WeatherPi weewx[1501]: restx: StationRegistry: > Registration not requested. > Jul 11 17:22:17 WeatherPi weewx[1501]: restx: Wunderground: Posting not > enabled. > Jul 11 17:22:17 WeatherPi weewx[1501]: restx: PWSweather: Posting not > enabled. > Jul 11 17:22:17 WeatherPi weewx[1501]: restx: CWOP: Posting not enabled. > Jul 11 17:22:17 WeatherPi weewx[1501]: restx: WOW: Posting not enabled. > Jul 11 17:22:17 WeatherPi weewx[1501]: restx: AWEKAS: Posting not enabled. > Jul 11 17:22:17 WeatherPi weewx[1501]: engine: Starting up weewx version > 3.9.1 > Jul 11 17:22:17 WeatherPi weewx[1501]: engine: Starting main packet loop. > Jul 11 17:22:35 WeatherPi weewx[1501]: engine: 2019-07-11 17:22:35 EDT > (1562880155) LOOP value 'inHumidity' nan outside limits (0.0, 100.0) > Jul 11 17:22:35 WeatherPi weewx[1501]: engine: 2019-07-11 17:22:35 EDT > (1562880155) LOOP value 'inTemp' nan outside limits (10.0, 120.0) > Jul 11 17:23:02 WeatherPi weewx[1501]: engine: 2019-07-11 17:23:03 EDT > (1562880183) LOOP value 'inHumidity' nan outside limits (0.0, 100.0) > Jul 11 17:23:02 WeatherPi weewx[1501]: engine: 2019-07-11 17:23:03 EDT > (1562880183) LOOP value 'inTemp' nan outside limits (10.0, 120.0) > Jul 11 17:23:30 WeatherPi weewx[1501]: engine: 2019-07-11 17:23:31 EDT > (1562880211) LOOP value 'inHumidity' nan outside limits (0.0, 100.0) > Jul 11 17:23:30 WeatherPi weewx[1501]: engine: 2019-07-11 17:23:31 EDT > (1562880211) LOOP value 'inTemp' nan outside limits (10.0, 120.0) > Jul 11 17:25:21 WeatherPi weewx[1501]: manager: Added record 2019-07-11 > 17:25:00 EDT (1562880300) to database 'weewx.sdb' > Jul 11 17:25:21 WeatherPi weewx[1501]: manager: Added record 2019-07-11 > 17:25:00 EDT (1562880300) to daily summary in 'weewx.sdb' > Jul 11 17:25:45 WeatherPi weewx[1501]: cheetahgenerator: Generated 8 files > for report SeasonsReport in 23.61 seconds > Jul 11 17:25:46 WeatherPi weewx[1501]: imagegenerator: Generated 14 images > for SeasonsReport in 1.11 seconds > Jul 11 17:25:46 WeatherPi weewx[1501]: copygenerator: copied 5 files to > /var/www/html/weewx > Jul 11 17:30:25 WeatherPi weewx[1501]: manager: Added record 2019-07-11 > 17:30:00 EDT (1562880600) to database 'weewx.sdb' > Jul 11 17:30:25 WeatherPi weewx[1501]: manager: Added record 2019-07-11 > 17:30:00 EDT (1562880600) to daily summary in 'weewx.sdb' > Jul 11 17:30:47 WeatherPi weewx[1501]: cheetahgenerator: Generated 8 files > for report SeasonsReport in 21.79 seconds > Jul 11 17:30:48 WeatherPi weewx[1501]: imagegenerator: Generated 14 images > for SeasonsReport in 1.10 seconds > Jul 11 17:30:48 WeatherPi weewx[1501]: copygenerator: copied 0 files to > /var/www/html/weewx > Jul 11 17:31:39 WeatherPi systemd[1]: Starting Cleanup of Temporary > Directories... > Jul 11 17:31:39 WeatherPi systemd[1]: Started Cleanup of Temporary > Directories. > > > On Thursday, July 11, 2019 at 4:21:03 PM UTC-4, Thomas Keffer wrote: >> >> Which version of FileParse are you using? >> >> What does the log say? >> >> -tk >> >> >> On Thu, Jul 11, 2019 at 12:52 PM 'Ron Walker' via weewx-user < >> [email protected]> wrote: >> >>> Hi All, >>> >>> I running WeeWX 3.9.1 on A Raspberry Pi 3 B+. I have set a loop >>> interval in the weewx.conf of 10 seconds, but looking at the incoming loop >>> records the interval appears to be 28-30 seconds. Is there a setting >>> somewhere else that would affect this? >>> >>> weewx.conf setting: >>> >>> [FileParse] >>> poll_interval = 10 >>> path = /var/tmp/data.csv >>> driver = user.fileparse >>> loop_interval = 10 >>> >>> loop data: >>> >>> LOOP: 2019-07-11 15:49:38 EDT (1562874578) windSpeed2avg: 0.0, 60.3: >>> 60.3, altimeter: 30.0268053867, appTemp: 83.3091122194, barometer: >>> 30.0215298055, boxHumdity: 41.0, boxTemp: 71.6, cloudbase: 460.0, dateTime: >>> 1562874578, dewpoint: 73.7, dGustDir10avg: 0.0, GustDir10avg: 45.0, >>> heatindex: 73.7, humidex: 92.484214186, inDewpoint: 50.2896337578, >>> inHumdity: 40.0, inHumidity: 47.0, inTemp: 71.6, maxSolarRad: None, >>> outHumidity: 100.0, outTemp: 73.7, pressure: 29.54, radiation: 0.18, rain: >>> 0.01, rainRate: 0.36, supplyVoltage: 4.35, ustDir10avg: 270.0, usUnits: 1, >>> windchill: 73.7, windDir: None, windDir2avg: 352.0, windGust: 1.5, >>> windGust10avg: 1.5, windGustDir: 0.0, windGustDir10avg: 0.0, windSpeed: >>> 0.0, windSpeed2avg: 0.0 >>> LOOP: 2019-07-11 15:50:07 EDT (1562874607) windSpeed2avg: 0.0, 60.3: >>> 60.3, altimeter: 30.0268053867, appTemp: 83.3091122194, barometer: >>> 30.0215298055, boxHumdity: 41.0, boxTemp: 71.6, cloudbase: 460.0, dateTime: >>> 1562874607, dewpoint: 73.7, dGustDir10avg: 0.0, GustDir10avg: 45.0, >>> heatindex: 73.7, humidex: 92.484214186, inDewpoint: 50.2896337578, >>> inHumdity: 40.0, inHumidity: 47.0, inTemp: 71.6, maxSolarRad: None, >>> outHumidity: 100.0, outTemp: 73.7, pressure: 29.54, radiation: 0.19, rain: >>> 0.01, rainRate: 0.4, supplyVoltage: 4.33, ustDir10avg: 270.0, usUnits: 1, >>> windchill: 73.7, windDir: None, windDir2avg: 346.0, windGust: 1.5, >>> windGust10avg: 1.5, windGustDir: 0.0, windGustDir10avg: 0.0, windSpeed: >>> 0.0, windSpeed2avg: 0.0 >>> LOOP: 2019-07-11 15:50:35 EDT (1562874635) windSpeed2avg: 0.0, 60.3: >>> 60.3, altimeter: 30.0268053867, appTemp: 83.1528623604, barometer: >>> 30.0216208352, boxHumdity: 41.0, boxTemp: 71.6, cloudbase: 460.0, dateTime: >>> 1562874635, dewpoint: 73.6, dGustDir10avg: 0.0, GustDir10avg: 45.0, >>> heatindex: 73.6, humidex: 92.2856972351, inDewpoint: 50.2896337578, >>> inHumdity: 40.0, inHumidity: 47.0, inTemp: 71.6, maxSolarRad: None, >>> outHumidity: 100.0, outTemp: 73.6, pressure: 29.54, radiation: 0.19, rain: >>> 0.01, rainRate: 0.44, supplyVoltage: 4.36, ustDir10avg: 270.0, usUnits: 1, >>> windchill: 73.6, windDir: None, windDir2avg: 340.0, windGust: 1.5, >>> windGust10avg: 1.5, windGustDir: 0.0, windGustDir10avg: 0.0, windSpeed: >>> 0.0, windSpeed2avg: 0.0 >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/029cfd2e-882b-4732-baad-baf9d09b3245%40googlegroups.com >>> <https://groups.google.com/d/msgid/weewx-user/029cfd2e-882b-4732-baad-baf9d09b3245%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/e6e5eb11-7f17-4f87-894d-def6618d475f%40googlegroups.com > <https://groups.google.com/d/msgid/weewx-user/e6e5eb11-7f17-4f87-894d-def6618d475f%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEAs7iX1WzF-CKr9mZijPZrNtViF7dZDJbVvbq9dt3B4cQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
#!/usr/bin/python # # Copyright 2014 Matthew Wall # # weewx driver that reads data from a file # # This program is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free Software # Foundation, either version 3 of the License, or any later version. # # This program is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS # FOR A PARTICULAR PURPOSE. # # See http://www.gnu.org/licenses/ # This driver will read data from a file. Each line of the file is a # name=value pair, for example: # # temperature=50 # humidity=54 # in_temperature=75 # # The units must be in the weewx.US unit system: # degree_F, inHg, inch, inch_per_hour, mile_per_hour # # To use this driver, put this file in the weewx user directory, then make # the following changes to weewx.conf: # # [Station] # station_type = FileParse # [FileParse] # poll_interval = 2 # number of seconds # path = /var/tmp/wxdata # location of data file # driver = user.fileparse # # If the variables in the file have names different from those in the database # schema, then create a mapping section called label_map. This will map the # variables in the file to variables in the database columns. For example: # # [FileParse] # ... # [[label_map]] # temp = outTemp # humi = outHumidity # in_temp = inTemp # in_humid = inHumidity from __future__ import with_statement import syslog import time import weewx.drivers from weeutil.weeutil import timestamp_to_string DRIVER_NAME = 'FileParse' DRIVER_VERSION = "0.6" def logmsg(dst, msg): syslog.syslog(dst, 'fileparse: %s' % msg) def logdbg(msg): logmsg(syslog.LOG_DEBUG, msg) def loginf(msg): logmsg(syslog.LOG_INFO, msg) def logerr(msg): logmsg(syslog.LOG_ERR, msg) def _get_as_float(d, s): v = None if s in d: try: v = float(d[s]) except ValueError as e: logerr("cannot read value for '%s': %s" % (s, e)) return v def loader(config_dict, engine): return FileParseDriver(**config_dict[DRIVER_NAME]) class FileParseDriver(weewx.drivers.AbstractDevice): """weewx driver that reads data from a file""" def __init__(self, **stn_dict): # where to find the data file self.path = stn_dict.get('path', '/var/tmp/wxdata') # how often to poll the weather data file, seconds self.poll_interval = float(stn_dict.get('poll_interval', 2.5)) # mapping from variable names to weewx names self.label_map = stn_dict.get('label_map', {}) loginf("data file is %s" % self.path) loginf("polling interval is %s" % self.poll_interval) loginf('label map is %s' % self.label_map) def genLoopPackets(self): while True: # read whatever values we can get from the file data = {} try: logdbg("Opening file %s at %s" % (self.path, timestamp_to_string(time.time()+0.5))) with open(self.path) as f: for line in f: eq_index = line.find('=') name = line[:eq_index].strip() value = line[eq_index + 1:].strip() data[name] = value logdbg("Closing file %s at %s" % (self.path, timestamp_to_string(time.time()+0.5))) except Exception as e: logerr("read failed: %s" % e) # map the data into a weewx loop packet _packet = {'dateTime': int(time.time() + 0.5), 'usUnits': weewx.US} for vname in data: _packet[self.label_map.get(vname, vname)] = _get_as_float(data, vname) logdbg("Yielding %s at %s" % (_packet['dateTime'], timestamp_to_string(time.time()+0.5))) yield _packet time.sleep(self.poll_interval) @property def hardware_name(self): return "FileParse" # To test this driver, run it directly as follows: # PYTHONPATH=/home/weewx/bin python /home/weewx/bin/user/fileparse.py if __name__ == "__main__": import weeutil.weeutil driver = FileParseDriver() for packet in driver.genLoopPackets(): print weeutil.weeutil.timestamp_to_string(packet['dateTime']), packet
