The source data is csv with file head: "dateTime", "SoC" 2023-07-18 06:32:00, 38.0 2023-07-18 06:32:10, 38.0 2023-07-18 06:32:20, 38.0 2023-07-18 06:32:30, 38.0 2023-07-18 06:32:40, 37.9 2023-07-18 06:32:50, 37.9 2023-07-18 06:33:00, 37.9 2023-07-18 06:33:10, 37.9 2023-07-18 06:33:30, 37.9 2023-07-18 06:33:40, 37.9
and import config file attached, which was modified from the example file. thanks On Tuesday, July 18, 2023 at 5:27:10 PM UTC+10 gjr80 wrote: > The error says it all. WeeWX field interval cannot be 0. What is in your > source data and your import config file? > > Gary > > > > On Tuesday, 18 July 2023 at 06:28:10 UTC+1 IceNov wrote: > > Thank you for the responses. I was able to get the MQTT data into the > database by creating a new field in the database as advised. But still no > success with importing the CSV. I suspect it is a datetime/interval issue > as I noticed lots of these in the logs: > > Jul 18 13:51:00 newmqtt wee_import[17507] INFO weewx.manager: Non-positive > value for record field 'interval': 0 > Jul 18 13:51:00 newmqtt wee_import[17507] INFO weewx.manager: *** record > ignored > Jul 18 13:51:00 newmqtt wee_import[17507] INFO weewx.manager: Added record > 2023-07-18 13:44:30 AEST (1689651870) to database 'weewx.sdb' > > I'll keep trying... > > Tony > On Monday, July 17, 2023 at 3:24:23 AM UTC+10 vince wrote: > > On Sunday, July 16, 2023 at 12:17:47 AM UTC-7 IceNov wrote: > > I can see the mqtt data when running weewxd from the command line - it is > one of the fields showing up in the terminal output - but it's not being > added to the database. > > I've also tried importing a csv file with timestamp and value - which > reported success - but still no new column in the db. > How do I get a new measurement into the standard database? > > > Fields don't magically appear in the database. You have to create them > yourself somehow. > > Simplest way is to map the incoming MQTT topic to an existing field in the > database. See the example at > https://github.com/bellrichm/WeeWX-MQTTSubscribe/wiki/individual-example > for help. Notice how topic/temp1 is mapped to extraTemp1 in the db ? > > > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/02eb9799-0250-4e0a-920b-dd89cb1aff29n%40googlegroups.com.
# EXAMPLE CONFIGURATION FILE FOR IMPORTING FROM CSV FILES # # Copyright (c) 2009-2022 Tom Keffer <[email protected]> and Gary Roderick. # See the file LICENSE.txt for your rights. ############################################################################## # Specify the source. Available options are: # CSV - import obs from a single CSV format file # WU - import obs from a Weather Underground PWS history # Cumulus - import obs from a one or more Cumulus monthly log files # WD - import obs from a one or more WD monthly log files # Format is: # source = (CSV | WU | Cumulus) source = CSV ############################################################################## [CSV] # Parameters used when importing from a CSV file # Path and name of our CSV source file. Format is: # file = full path and filename file = /home/pi/weewx-4.10.2/soc.csv # The character used to separate fields. Format is: # delimiter = <single character> # Default is , (comma). delimiter = ',' # Specify the character used as the decimal point. The character # must be enclosed in quotes. # Format is: # decimal = '.' (dot) # or # decimal = ',' (comma) decimal = '.' # If there is no mapped interval field how will the interval field be # determined for the imported records. Available options are: # derive - Derive the interval field from the timestamp of successive # records. This setting is best used when there are no missing # records from period being imported. Missing records will # cause the interval field to be incorrectly calculated for some # records. # conf - Use the interval setting from weewx.conf. This setting is # best used if the records to be imported have been produced by # WeeWX or some other means with the same archive interval as # set in weewx.conf on this machine. # x - Use a fixed interval of 'x' minutes for every record where 'x' # is a number. This setting is best used if the records to be # imported are equally spaced in time but there are some missing # records. # # Note: If there is a mapped interval field then this setting will be # ignored. # Format is: # interval = (derive | conf | x) interval = derive # Should the [StdQC] max/min limits in weewx.conf be applied to the # imported data. This may be useful if the source has extreme values that # are clearly incorrect for some observations. Available options are: # True - weewx.conf [StdQC] max/min limits are applied. # False - weewx.conf [StdQC] max/min limits are not applied. # Format is: # qc = (True | False) qc = False # Should any missing derived observations be calculated from the imported # data if possible. Available options are: # True - Any missing derived observations are calculated. # False - Any missing derived observations are not calculated. # Format is: # calc_missing = (True | False) calc_missing = False # Specify how imported data fields that contain invalid data (eg a numeric # field containing non-numeric data) are handled. Available options are: # True - The invalid data is ignored, the WeeWX target field is set to # None and the import continues. # False - The import is halted. # Format is: # ignore_invalid_data = (True | False) # Default is True. ignore_invalid_data = True # Imported records are written to archive in transactions of tranche # records at a time. Increase for faster throughput, decrease to reduce # memory requirements. Format is: # tranche = x # where x is an integer tranche = 250 # Specify whether a UV sensor was used to produce any UV observations. # Available options are: # True - UV sensor was used and UV data will be imported. # False - UV sensor was not used and any UV data will not be imported. # UV fields will be set to None/NULL. # For a CSV import UV_sensor should be set to False if a UV sensor was # NOT present when the import data was created. Otherwise it may be set to # True or omitted. Format is: # UV_sensor = (True | False) UV_sensor = False # Specify whether a solar radiation sensor was used to produce any solar # radiation observations. Available options are: # True - Solar radiation sensor was used and solar radiation data will # be imported. # False - Solar radiation sensor was not used and any solar radiation # data will not be imported. radiation fields will be set to # None/NULL. # For a CSV import solar_sensor should be set to False if a solar radiation # sensor was NOT present when the import data was created. Otherwise it may # be set to True or omitted. Format is: # solar_sensor = (True | False) solar_sensor = False # Date-time format of CSV field from which the WeeWX archive record # dateTime field is to be extracted. wee_import first attempts to interpret # date/time info in this format, if this fails it then attempts to # interpret it as a timestamp and if this fails it then raises an error. # Uses Python strptime() format codes. # raw_datetime_format = Python strptime() format string raw_datetime_format = %Y-%m-%d %H:%M:%S # Does the imported rain field represent the total rainfall since the last # record or a cumulative value. Available options are: # discrete - rain field represents total rainfall since last record # cumulative - rain field represents a cumulative rainfall reset at # midnight # rain = (discrete | cumulative) rain = cumulative # Lower and upper bounds for imported wind direction. It is possible, # particularly for a calculated direction, to have a value (eg -45) outside # of the WeeWX limits (0 to 360 inclusive). Format is: # # wind_direction = lower,upper # # where : # lower is the lower limit of acceptable wind direction in degrees # (may be negative) # upper is the upper limit of acceptable wind direction in degrees # # Imported values from lower to upper will be normalised to the range 0 to # 360. Values outside of the parameter range will be stored as None. # Default is -360,360. wind_direction = -360,360 # Map CSV record fields to WeeWX archive fields. Format is: # # weewx_archive_field_name = csv_field_name, weewx_unit_name # # where: # weewx_archive_field_name - An observation name in the WeeWX database # schema. # csv_field_name - The name of a field from the CSV file. # weewx_unit_name - The name of the units, as defined in WeeWX, # used by csv_field_name. wee_import will do # the necessary conversions to the unit system # used by the WeeWX archive. # For example, # outTemp = Temp, degree_C # would map the CSV field Temp, in degrees C, to the archive field outTemp. # # A mapping for WeeWX field dateTime is mandatory and the WeeWX unit name # for the dateTime mapping must be unix_epoch. For example, # dateTime = csv_date_and_time, unix_epoch # would map the CSV field csv_date_and_time to the WeeWX dateTime field with # the csv_date_and_time field being interpreted first using the format # specified at the raw_datetime_format config option and if that fails as a # unix epoch timestamp. # # Field mapping to the WeeWX usUnits archive field is currently not # supported. If a usUnits field exists in the CSV data it should not be # mapped, rather WeeWX unit names should included against each field to be # imported in the field map. # # WeeWX archive fields that do not exist in the CSV data may be omitted. # Any omitted fields that are derived (eg dewpoint) may be calculated # during import using the equivalent of the WeeWX StdWXCalculate service # through setting the calc-missing parameter above. [[FieldMap]] dateTime = dateTime, unix_epoch interval = barometer = barometer, inHg pressure = altimeter = inTemp = outTemp = Temp, degree_F inHumidity = outHumidity = humidity, percent windSpeed = windspeed, mile_per_hour windDir = wind, degree_compass windGust = gust, mile_per_hour windGustDir = gustDir, degree_compass rainRate = rate, inch_per_hour rain = dayrain, inch dewpoint = windchill = heatindex = ET = radiation = UV = SoC = snow, percent
