I've just run wee_import using your CSV data and import config file and had no problems with 'time', but encountered errors on almost every other field. barometer/Barometer needs to be fixed; you either need to change the field name in your CSV data or change the input field name in the [[FieldMap]] in your import config file - either way they need to match. Also, you need to remove the trailing spaces from the field names in the header line of your CSV data file - the python reader object used to parse the CSV data file does not trim trailing spaces.
I presume also that the single quote character terminating the UV line in your [[FieldMap]] stanza is in fact a copy/paste artefact and does not actually appear in the import config file. Make the above changes and try the import again. If it still fails look very closely at your CSV data, are there any characters preceding the 'time' on the header line? There could be some sort of non-visible character before 'time' so try copying the CSV data to a new file and then manually typing in a new header line, save the file and use that as your source data. Gary On Wednesday, 26 July 2023 at 15:43:45 UTC+1 [email protected] wrote: > I have a problem with wee_import > > This is CSV file > > time;Barometer;Temp ;High Temp ;Low Temp ;Hum;Dew Point ;Wet Bulb ;Wind > Speed ;Wind Direction;Wind Run - km;High Wind Speed ;High Wind > Direction;Wind Chill ;Heat Index ;THW Index ;Rain ;Rain Rate ;ET ;Heating > Degree Days;Cooling Degree Days > 02/08/2022 > 15:30;1015,0;28,9;29,2;28,8;43,0;15,1;18,1;16,1;N;4,02;27,4;NNE;28,3;28,7;28,1;0,0;0,0;0,00;0,000;0,110 > > > > This is configuration CSV.conf > > `# EXAMPLE CONFIGURATION FILE FOR IMPORTING FROM CSV FILES > Copyright (c) 2009-2022 Tom Keffer [email protected] and Gary Roderick.See > the file LICENSE.txt for your rights. > > > ############################################################################## > Specify the source. Available options are:CSV - import obs from a single > CSV format fileWU - import obs from a Weather Underground PWS historyCumulus > - import obs from a one or more Cumulus monthly log filesWD - import obs > from a one or more WD monthly log filesFormat is:source = (CSV | WU | > Cumulus) > > source = CSV > > > ############################################################################## > > [CSV] > # Parameters used when importing from a CSV file > # Path and name of our CSV source file. Format is: # file = full path and > filename file = /etc/weewx/martina.csv # The character used to separate > fields. Format is: # delimiter = <single character> # Default is , (comma). > delimiter = ';' # Specify the character used as the decimal point. The > character # must be enclosed in quotes. # Format is: # decimal = '.' (dot) > # or # decimal = ',' (comma) decimal = ',' # If there is no mapped interval > field how will the interval field be # determined for the imported records. > Available options are: # derive - Derive the interval field from the > timestamp of successive # records. This setting is best used when there are > no missing # records from period being imported. Missing records will # > cause the interval field to be incorrectly calculated for some # records. # > conf - Use the interval setting from weewx.conf. This setting is # best > used if the records to be imported have been produced by # WeeWX or some > other means with the same archive interval as # set in weewx.conf on this > machine. # x - Use a fixed interval of 'x' minutes for every record where > 'x' # is a number. This setting is best used if the records to be # > imported are equally spaced in time but there are some missing # records. # > # Note: If there is a mapped interval field then this setting will be # > ignored. # Format is: # interval = (derive | conf | x) interval = conf # > Should the [StdQC] max/min limits in weewx.conf be applied to the # > imported data. This may be useful if the source has extreme values that # > are clearly incorrect for some observations. Available options are: # True > - weewx.conf [StdQC] max/min limits are applied. # False - weewx.conf > [StdQC] max/min limits are not applied. # Format is: # qc = (True | False) > qc = True # Should any missing derived observations be calculated from the > imported # data if possible. Available options are: # True - Any missing > derived observations are calculated. # False - Any missing derived > observations are not calculated. # Format is: # calc_missing = (True | > False) calc_missing = True # Specify how imported data fields that contain > invalid data (eg a numeric # field containing non-numeric data) are > handled. Available options are: # True - The invalid data is ignored, the > WeeWX target field is set to # None and the import continues. # False - The > import is halted. # Format is: # ignore_invalid_data = (True | False) # > Default is True. ignore_invalid_data = True # Imported records are written > to archive in transactions of tranche # records at a time. Increase for > faster throughput, decrease to reduce # memory requirements. Format is: # > tranche = x # where x is an integer tranche = 250 # Specify whether a UV > sensor was used to produce any UV observations. # Available options are: # > True - UV sensor was used and UV data will be imported. # False - UV sensor > was not used and any UV data will not be imported. # UV fields will be set > to None/NULL. # For a CSV import UV_sensor should be set to False if a UV > sensor was # NOT present when the import data was created. Otherwise it may > be set to # True or omitted. Format is: # UV_sensor = (True | False) > UV_sensor = True # Specify whether a solar radiation sensor was used to > produce any solar # radiation observations. Available options are: # True - > Solar radiation sensor was used and solar radiation data will # be > imported. # False - Solar radiation sensor was not used and any solar > radiation # data will not be imported. radiation fields will be set to # > None/NULL. # For a CSV import solar_sensor should be set to False if a > solar radiation # sensor was NOT present when the import data was created. > Otherwise it may # be set to True or omitted. Format is: # solar_sensor = > (True | False) solar_sensor = True # Date-time format of CSV field from > which the WeeWX archive record # dateTime field is to be extracted. > wee_import first attempts to interpret # date/time info in this format, if > this fails it then attempts to # interpret it as a timestamp and if this > fails it then raises an error. # Uses Python strptime() format codes. # > raw_datetime_format = Python strptime() format string raw_datetime_format = > %d/%m/%Y %H:%M # Does the imported rain field represent the total rainfall > since the last # record or a cumulative value. Available options are: # > discrete - rain field represents total rainfall since last record # > cumulative - rain field represents a cumulative rainfall reset at # > midnight # rain = (discrete | cumulative) rain = discrete # Lower and upper > bounds for imported wind direction. It is possible, # particularly for a > calculated direction, to have a value (eg -45) outside # of the WeeWX > limits (0 to 360 inclusive). Format is: # # wind_direction = lower,upper # > # where : # lower is the lower limit of acceptable wind direction in > degrees # (may be negative) # upper is the upper limit of acceptable wind > direction in degrees # # Imported values from lower to upper will be > normalised to the range 0 to # 360. Values outside of the parameter range > will be stored as None. # Default is -360,360. wind_direction = -360,360 # > Map CSV record fields to WeeWX archive fields. Format is: # # > weewx_archive_field_name = csv_field_name, weewx_unit_name # # where: # > weewx_archive_field_name - An observation name in the WeeWX database # > schema. # csv_field_name - The name of a field from the CSV file. # > weewx_unit_name - The name of the units, as defined in WeeWX, # used by > csv_field_name. wee_import will do # the necessary conversions to the unit > system # used by the WeeWX archive. # For example, # outTemp = Temp, > degree_C # would map the CSV field Temp, in degrees C, to the archive field > outTemp. # # A mapping for WeeWX field dateTime is mandatory and the WeeWX > unit name # for the dateTime mapping must be unix_epoch. For example, # > dateTime = csv_date_and_time, unix_epoch # would map the CSV field > csv_date_and_time to the WeeWX dateTime field with # the csv_date_and_time > field being interpreted first using the format # specified at the > raw_datetime_format config option and if that fails as a # unix epoch > timestamp. # # Field mapping to the WeeWX usUnits archive field is > currently not # supported. If a usUnits field exists in the CSV data it > should not be # mapped, rather WeeWX unit names should included against > each field to be # imported in the field map. # # WeeWX archive fields that > do not exist in the CSV data may be omitted. # Any omitted fields that are > derived (eg dewpoint) may be calculated # during import using the > equivalent of the WeeWX StdWXCalculate service # through setting the > calc-missing parameter above. [[FieldMap]] dateTime = time, unix_epoch > interval = barometer = barometer, hPa pressure = altimeter = inTemp = > outTemp = Temp, degree_C inHumidity = outHumidity = Hum, percent windSpeed > = Wind Speed, km_per_hour windDir = Wind Direction, degree_compass windGust > = High Wind Speed, km_per_hour windGustDir = High Wind Direction, > degree_compass rainRate = Rain Rate, mm_per_hour rain = Rain, mm dewpoint = > Dew Point, degree_C windchill = Wind Chill, degree_C heatindex = Heat > Index, degree_C ET = ET, mm radiation = UV =` > > I get this error: > > sudo wee_import --import-config=/etc/weewx/csv.conf > /usr/share/weewx/wee_import:719: DeprecationWarning: The distutils package > is deprecated and slated for removal in Python 3.12. Use setuptools or > check PEP 632 for potential alternatives > from distutils.version import StrictVersion > Using WeeWX configuration file /etc/weewx/weewx.conf > Starting wee_import... > A CSV import from source file '/etc/weewx/martina.csv' has been requested. > Using database binding 'wx_binding', which is bound to database 'weewx.sdb' > Destination table 'archive' unit system is '0x01' (US). > Missing derived observations will be calculated. > Starting import ... > **** Unable to map source data. > **** Field 'time' not found in source data. > **** Nothing done, exiting. > > Where is the problem? Thanks for your help? > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/099181ed-7b5b-4e0b-830f-c10ee570d39dn%40googlegroups.com.
