I could be wrong, but the raw_datetime_format is the datetime format in the CSV file (i.e. %m/%d%Y). The dateTime = timestamp, unix_epoch in the CSV Map basically means convert the timestamp column into unix_epoch time format.
On Tuesday, June 18, 2019 at 10:06:49 PM UTC-4, Andrew Milner wrote: > > shouldn't it just be dateTime = timestamp and the format of timestamp be > specified in raw_datetime_format ?? > > > > On Wednesday, 19 June 2019 04:32:30 UTC+3, rich T wrote: >> >> I'm trying to import a large CSV file into the database and I'm receiving >> the following error the the dry run. >> >> pi@raspberrypi:/home/weewx/bin $ sudo ./wee_import >> --import-config=/home/pi/Desktop/clog/csv.conf --dry-run >> Starting wee_import... >> A CSV import from source file >> '/home/pi/Desktop/clog/archive_updated1.csv' has been requested. >> Using database binding 'wx_binding', which is bound to database >> 'weewx.sdb' >> Destination table 'archive' unit system is '0x01' (US). >> Missing derived observations will be calculated. >> All WeeWX UV fields will be set to None. >> All WeeWX radiation fields will be set to None. >> This is a dry run, imported data will not be saved to archive. >> Traceback (most recent call last): >> File "./wee_import", line 703, in <module> >> main() >> File "./wee_import", line 653, in main >> source_obj.run() >> File "/home/weewx/bin/weeimport/weeimport.py", line 350, in run >> _mapped_data = self.mapRawData(_raw_data, self.archive_unit_sys) >> File "/home/weewx/bin/weeimport/weeimport.py", line 721, in mapRawData >> self.map[_field]['units'], >> KeyError: 'units' >> >> Here is the mapping in the csv.conf: >> >> [[FieldMap]] >> dateTime = timestamp, unix_epoch >> usUnits = units >> interval = interval >> barometer = barometer, inHg >> pressure = pressure >> altimeter = altimeter >> inTemp = inTemp >> outTemp = outTemp, degree_F >> inHumidity = inHumidity >> outHumidity = outHumidity, percent >> windSpeed = windSpeed, mile_per_hour >> windDir = windDir, degree_compass >> windGust = windGust, mile_per_hour >> windGustDir = windGustDir, degree_compass >> rainRate = rainRate, inch_per_hour >> rain = rain, inch >> dewpoint = dewpoint >> windchill = windchill >> heatindex = heatindex >> ET = >> radiation = >> UV = >> rxCheckPercent = rxCheckPercent >> >> Here is a small portion of the CSV file: >> >> >> timestamp,units,interval,barometer,pressure,altimeter,inTemp,outTemp,inHumidity,outHumidity,windSpeed,windDir,windGust,windGustDir,rainRate,rain,dewpoint,windchill,heatindex,ET,radiation,UV,extraTemp1,extraTemp2,extraTemp3,soilTemp1,soilTemp2,soilTemp3,soilTemp4,leafTemp1,leafTemp2,extraHumid1,extraHumid2,soilMoist1,soilMoist2,soilMoist3,soilMoist4,leafWet1,leafWet2,rxCheckPercent,txBatteryStatus,consBatteryVoltage,hail,hailRate,heatingTemp,heatingVoltage,supplyVoltage,referenceVoltage,windBatteryStatus,rainBatteryStatus,outTempBatteryStatus,inTempBatteryStatus >> 04/09/2018 >> 00:05,1,5,30.101,,,77.2,24.2,20,62,0,,0,,0,0,13.08666521,24.2,24.2,0,,,,,,,,,,,,,,,,,,,,99.9375,,,,,,,,,,,, >> 04/09/2018 >> 00:10,1,5,30.102,,,77,24,20,63,0,,0,,0,0,13.25922371,24,24,0,,,,,,,,,,,,,,,,,,,,97.375,,,,,,,,,,,, >> 04/09/2018 >> 00:15,1,5,30.108,,,76.9,23.8,20,63,0,,0,,0,0,13.06932323,23.8,23.8,0,,,,,,,,,,,,,,,,,,,,96.52083333,,,,,,,,,,,, >> 04/09/2018 >> 00:20,1,5,30.11,,,76.7,23.7,20,63,0,,0,,0,0,12.97437126,23.7,23.7,0,,,,,,,,,,,,,,,,,,,,97.375,,,,,,,,,,,, >> 04/09/2018 >> 00:25,1,5,30.112,,,76.6,23.5,20,63,0,,0,,0,0,12.78446384,23.5,23.5,0,,,,,,,,,,,,,,,,,,,,77.72916667,,,,,,,,,,,, >> 04/09/2018 >> 00:30,1,5,30.11,,,76.4,23.5,21,63,0,,0,,0,0,12.78446384,23.5,23.5,0,,,,,,,,,,,,,,,,,,,,99.9375,,,,,,,,,,,, >> 04/09/2018 >> 00:35,1,5,30.111,,,76.7,23.3,20,64,0,,3,225,0,0,12.95043454,23.3,23.3,0,,,,,,,,,,,,,,,,,,,,99.08333333,,,,,,,,,,,, >> 04/09/2018 >> 00:40,1,5,30.106,,,76.9,23.3,20,64,0,,0,,0,0,12.95043454,23.3,23.3,0,,,,,,,,,,,,,,,,,,,,99.08333333,,,,,,,,,,,, >> 04/09/2018 >> 00:45,1,5,30.11,,,77,23.1,20,64,0,,0,,0,0,12.76018685,23.1,23.1,0,,,,,,,,,,,,,,,,,,,,99.08333333,,,,,,,,,,,, >> 04/09/2018 >> 00:50,1,5,30.114,,,77,23,20,65,0,,0,,0,0,13.01554258,23,23,0,,,,,,,,,,,,,,,,,,,,100,,,,,,,,,,,, >> >> >> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/dfbab94e-0eee-410b-8d24-6bc471d2fc6b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
