No, commas are fine. If the commas were the problem wee_import would be failing hard. Let me finish working through your data.
Gary On Sunday, 15 January 2023 at 08:40:05 UTC+10 [email protected] wrote: > I think I should add a period to all values, not a comma. > > David Schulz schrieb am Samstag, 14. Januar 2023 um 23:35:21 UTC+1: > >> There should actually be values in there. >> >> echo "SELECT dateTime, rain FROM archive" | sqlite3 >> /var/lib/weewx/weewx.sdb >> 1546297200| >> 1546297260| >> 1546297320|0.0 >> 1546297380|0.0 >> >> DatumZeit;Temp;Feuchte;Luftdruck;Regen;Wind;Richtung;Windboeen >> 01.01.2019 00:00;6,9;97;1026,6;0,2;4,7;242;9,7 >> 01.01.2019 00:01;6,9;97;1026,6;0,2;4,7;251;7,9 >> 01.01.2019 00:02;6,9;97;1026,6;0;4,7;250;7,9 >> 01.01.2019 00:03;6,9;97;1026,6;0;4,7;245;4,7 >> >> >> gjr80 schrieb am Samstag, 14. Januar 2023 um 23:19:12 UTC+1: >> >>> OK. Let's go straight to the horses head, what happens when you manually >>> query the database archive table for the period concerned. Do each of the >>> rows show the correct rainfall data or is it missing? >>> >>> Gary >>> >>> On Sunday, 15 January 2023 at 08:06:20 UTC+10 [email protected] >>> wrote: >>> >>>> Thanks for the info. But for testing I delete every time the whole DB >>>> and the NOAA reports. I have a Weewx test environment specifically for >>>> this. >>>> >>>> gjr80 schrieb am Samstag, 14. Januar 2023 um 23:03:41 UTC+1: >>>> >>>>> You need to be very careful when using the NOAA format reports to >>>>> verify the import of historical data, this is because only the current >>>>> month and current year reports are updated each report cycle. Historical >>>>> reports are not updated so will not reflect and changes to earlier data. >>>>> The sample data you provided is from January 2019 so I am guessing you >>>>> did >>>>> not import into the current month and year on your system. Try deleting >>>>> all >>>>> of your NOAA format reports from your WeeWX system, they will all be >>>>> re-generated on the next report cycle based on the current database data. >>>>> >>>>> Gary >>>>> >>>>> On Sunday, 15 January 2023 at 07:06:08 UTC+10 [email protected] >>>>> wrote: >>>>> >>>>>> [image: 2023-01-14 22_05_47-Blub.png] >>>>>> >>>>>> David Schulz schrieb am Samstag, 14. Januar 2023 um 22:04:28 UTC+1: >>>>>> >>>>>>> wee_import --verbose --import-config=/etc/weewx/import/csv.conf >>>>>>> /usr/share/weewx/wee_import:719: DeprecationWarning: The distutils >>>>>>> package is deprecated and slated for removal in Python 3.12. Use >>>>>>> setuptools >>>>>>> or check PEP 632 for potential alternatives >>>>>>> from distutils.version import StrictVersion >>>>>>> Using WeeWX configuration file /etc/weewx/weewx.conf >>>>>>> Starting wee_import... >>>>>>> A CSV import from source file '/etc/weewx/import/EXP201901A.CSV' has >>>>>>> been requested. >>>>>>> The following options will be used: >>>>>>> config=/etc/weewx/weewx.conf, >>>>>>> import-config=/etc/weewx/import/csv.conf >>>>>>> source=/etc/weewx/import/EXP201901A.CSV, from=None, to=None >>>>>>> dry-run=None, calc_missing=True, ignore_invalid_data=True >>>>>>> tranche=1000, interval=derive, date/time_string_format=%d.%m.%Y >>>>>>> %H:%M >>>>>>> delimiter=';', rain=cumulative, wind_direction=[-360.0, 360.0] >>>>>>> UV=False, radiation=False >>>>>>> Using database binding 'wx_binding', which is bound to database >>>>>>> 'weewx.sdb' >>>>>>> Destination table 'archive' unit system is '0x01' (US). >>>>>>> Missing derived observations will be calculated. >>>>>>> All WeeWX UV fields will be set to None. >>>>>>> All WeeWX radiation fields will be set to None. >>>>>>> Starting import ... >>>>>>> Obtaining raw import data for period 1 ... >>>>>>> The following imported field-to-WeeWX field map will be used: >>>>>>> source field 'DatumZeit' in units 'unix_epoch' --> WeeWX field >>>>>>> 'dateTime' >>>>>>> source field 'Temp' in units 'degree_C' --> WeeWX field >>>>>>> 'outTemp' >>>>>>> source field 'Feuchte' in units 'percent' --> WeeWX field >>>>>>> 'outHumidity' >>>>>>> source field 'Luftdruck' in units 'hPa' --> WeeWX field >>>>>>> 'barometer' >>>>>>> source field 'Regen' in units 'mm' --> WeeWX field 'rain' >>>>>>> source field 'Wind' in units 'km_per_hour' --> WeeWX field >>>>>>> 'windSpeed' >>>>>>> source field 'Richtung' in units 'degree_compass' --> WeeWX >>>>>>> field 'windDir' >>>>>>> source field 'Windboeen' in units 'km_per_hour' --> WeeWX field >>>>>>> 'windGust' >>>>>>> Raw import data read successfully for period 1. >>>>>>> Mapping raw import data for period 1 ... >>>>>>> Mapped 1440 records. >>>>>>> Raw import data mapped successfully for period 1. >>>>>>> Saving mapped data to archive for period 1 ... >>>>>>> 1440 records identified for import. >>>>>>> Proceeding will save all imported records in the WeeWX archive. >>>>>>> Are you sure you want to proceed (y/n)? y >>>>>>> Unique records processed: 1440; Last timestamp: 2019-01-01 23:59:00 >>>>>>> CET (1546383540) >>>>>>> Mapped data saved to archive successfully ""for period 1. >>>>>>> Calculating missing derived observations ... >>>>>>> Processing record: 1440; Last record: 2019-01-02 00:00:00 CET >>>>>>> (1546383600) >>>>>>> Recalculating daily summaries... >>>>>>> Records processed: 1000; time: 2019-01-01 16:39:00 CET (1546357140) >>>>>>> Finished recalculating daily summaries >>>>>>> Finished calculating missing derived observations >>>>>>> Finished import >>>>>>> 1440 records were processed and 1440 unique records imported in >>>>>>> 40.49 seconds. >>>>>>> Those records with a timestamp already in the archive will not have >>>>>>> been >>>>>>> imported. Confirm successful import in the WeeWX log file. >>>>>>> >>>>>>> David Schulz schrieb am Samstag, 14. Januar 2023 um 22:03:34 UTC+1: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> i want to import some old weatherdata via csv file, but after the >>>>>>>> import, the raindata is missing. i tried the import with "rain = >>>>>>>> cumulative >>>>>>>> / discrete". >>>>>>>> >>>>>>>> Here my config: >>>>>>>> >>>>>>>> source = CSV >>>>>>>> ####################################### >>>>>>>> [CSV] >>>>>>>> file = /etc/weewx/import/EXP201901A.CSV >>>>>>>> delimiter = ; >>>>>>>> interval = derive >>>>>>>> qc = True >>>>>>>> calc_missing = True >>>>>>>> ignore_invalid_data = True >>>>>>>> tranche = 1000 >>>>>>>> UV_sensor = False >>>>>>>> solar_sensor = False >>>>>>>> raw_datetime_format = %d.%m.%Y %H:%M >>>>>>>> # rain = (discrete | cumulative) >>>>>>>> rain = cumulative >>>>>>>> wind_direction = -360,360 >>>>>>>> [[FieldMap]] >>>>>>>> dateTime = DatumZeit, unix_epoch >>>>>>>> outTemp = Temp, degree_C >>>>>>>> outHumidity = Feuchte, percent >>>>>>>> barometer = Luftdruck, hPa >>>>>>>> rain = Regen, mm >>>>>>>> windSpeed = Wind, km_per_hour >>>>>>>> windDir = Richtung, degree_compass >>>>>>>> windGust = Windboeen, km_per_hour >>>>>>>> >>>>>>>> And here some sample data. >>>>>>>> >>>>>>>> DatumZeit;Temp;Feuchte;Luftdruck;Regen;Wind;Richtung;Windboeen >>>>>>>> 01.01.2019 00:00;6,9;97;1026,6;0,2;4,7;242;9,7 >>>>>>>> 01.01.2019 00:01;6,9;97;1026,6;0,2;4,7;251;7,9 >>>>>>>> 01.01.2019 00:02;6,9;97;1026,6;0,2;4,7;250;7,9 >>>>>>>> 01.01.2019 00:03;6,9;97;1026,6;0,2;4,7;245;4,7 >>>>>>>> 01.01.2019 00:04;6,9;97;1026,6;0;4,7;250;7,9 >>>>>>>> 01.01.2019 00:05;6,9;97;1026,5;0;4,7;234;7,9 >>>>>>>> 01.01.2019 00:06;6,9;97;1026,5;0;4,7;232;11,2 >>>>>>>> 01.01.2019 00:07;7;97;1026,5;0;4,7;234;4,7 >>>>>>>> 01.01.2019 00:08;7;97;1026,5;0;4,7;241;7,9 >>>>>>>> 01.01.2019 00:09;7;97;1026,5;0;4,7;240;6,5 >>>>>>>> >>>>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/386d41ae-3ca3-47ca-8cc2-217efd7e49d2n%40googlegroups.com.
