Thanks for your answer.
The two conditions you listed are satisfied: the necessary other values are
present in my csv, and no records were already existing for such period in
the database.
Answering your questions:
- Those are my data (a sample):
CSV data (more fields than used in the import process are present)
dateTime,outsideTemp,hiOutsideTemp,lowOutsideTemp,outsideHum,windSpeed,windDirection,hiWindSpeed,hiWindDirection,barometer,rain,hiRainRate,insideTemp,insideHum,numWindSamples,archiveInterval
2021-08-01T00:01,27.61,27.61,27.61,60.0,1.61,SW,3.22,SW,1006.30,0.00,0.00,30.33,49.0,22,1
2021-08-01T00:02,27.56,27.61,27.56,60.0,1.61,W,1.61,SW,1006.16,0.00,0.00,30.33,49.0,24,1
2021-08-01T00:03,27.50,27.56,27.50,60.0,1.61,W,1.61,W,1006.27,0.00,0.00,30.33,49.0,23,1
2021-08-01T00:04,27.50,27.50,27.50,60.0,0.00,SW,3.22,SW,1006.33,0.00,0.00,30.33,49.0,22,1
2021-08-01T00:05,27.50,27.50,27.50,60.0,3.22,WSW,4.83,WSW,1006.37,0.00,0.00,30.33,49.0,24,1
Resulting db records (extracted querying the `archive`. I here removed
all null values for reading convenience)
dateTime,usUnits,interval,appTemp,barometer,cloudbase,humidex,inDewpoint,inHumidity,inTemp,outHumidity,outTemp,rain,rainRate,windDir,windGust,windGustDir,windrun,windSpeed
1627768860,16,1,30.5895137231874,1006.3,1167.77458102164,34.4779454220626,18.4017421866142,49,30.33,60,27.61,0,0,225,3.22,225,0.0268333333333333,1.61
1627768920,16,1,30.5182757269146,1006.16,1167.3911940789,34.3910793180734,18.4017421866142,49,30.33,60,27.56,0,0,270,1.61,225,0.0268333333333333,1.61
1627768980,16,1,30.4328612401526,1006.27,1166.93122058363,34.286968264113,18.4017421866142,49,30.33,60,27.5,0,0,270,1.61,270,0.0268333333333333,1.61
1627769040,16,1,30.7459167959586,1006.33,1166.93122058363,34.286968264113,18.4017421866142,49,30.33,60,27.5,0,0,,3.22,225,0,0
1627769100,16,1,30.1198056843466,1006.37,1166.93122058363,34.286968264113,18.4017421866142,49,30.33,60,27.5,0,0,247.5,4.83,247.5,0.0536666666666667,3.22
- No data were already present for that period
- I checked resulting records by querying the database directly
- The query I ran (for the sample): SELECT * FROM archive WHERE dateTime >=
1627768860 and dateTime < 1627769160 ORDER BY dateTime
- I imported data few weeks ago, so I cannot exactly say what the output of
the import command was, but - as I can remember - nothing remarkable was
printed out. I noticed just now that some fields are missing. Can I still
check those logs? how to?
Thanks,
Andrea
Il giorno domenica 7 agosto 2022 alle 02:09:20 UTC+2 gjr80 ha scritto:
> Your import config file is fine and provided a couple of conditions were
> met the missing fields should have been calculated. The two conditions are
> (1) any pre-requisites for calculating the fields concerned must exist, for
> example for heatindex, windchill and dewpoint you must have a non-None
> outTemp value. If a pre-requisite is missing the calculated value will be
> None. (2) Even if the calculated values were calculated there must be no
> record already existing in the archive with the same timestamp as the
> record being imported. If such a record already exists in the archive the
> imported record is discarded in it's entirety (not just the calculated
> fields). So if these conditions were met your imported data including
> calculated fields should have been imported and saved to archive, if they
> were not imported then something went wrong.
>
> A few questions to try to narrow down the cause. What does your data look
> like? Can you post some of the records here or if you don't want to can you
> reply privately to this post with your raw import data. Did records already
> exist in your archive for the period covered by the data you were
> importing? How did you check the imported data? Did you query the database
> archive table directly or rely on WeeWX generated output? If the former
> what command did you use and what results were presented? Did the other
> non-calculated fields import correctly? What was presented on-screen when
> you ran wee_import? Were there any errors or warnings? wee-import will
> have written some entries to the log at the time of the import, they should
> be clearly evident with a 'wee_import' label, what do these entries say?
>
> Gary
>
> On Sunday, 7 August 2022 at 00:23:31 UTC+10 [email protected] wrote:
>
>> I imported ~1 year of data from csv, using this `import.conf`
>> configuration:
>> source = CSV
>> [CSV]
>> file = my-data.csv
>> interval = 1
>> qc = True
>> calc_missing = True
>> ignore_invalid_data = True
>> tranche = 250
>> UV_sensor = False
>> solar_sensor = False
>> raw_datetime_format = %Y-%m-%dT%H:%M
>> rain = discrete
>> wind_direction = 0,360
>> [[FieldMap]]
>> dateTime = dateTime, unix_epoch
>> usUnits =
>> interval =
>> barometer = barometer, hPa
>> pressure =
>> altimeter =
>> inTemp = insideTemp, degree_C
>> outTemp = outsideTemp, degree_C
>> inHumidity = insideHum, percent
>> outHumidity = outsideHum, percent
>> windSpeed = windSpeed, km_per_hour
>> windDir = windDirection, degree_compass
>> windGust = hiWindSpeed, km_per_hour
>> windGustDir = hiWindDirection, degree_compass
>> rainRate = hiRainRate, mm_per_hour
>> rain = rain, mm
>> dewpoint =
>> windchill =
>> heatindex =
>> ET =
>> radiation =
>> UV =
>>
>> What I expected was that the following fields would have been calculated,
>> starting from the other available fields:
>> altimeter
>> dewpoint
>> heatindex
>> pressure
>> windchill
>> But in the `archive` table, those columns are empty.
>>
>> Did I do something wrong, or is the behavior correct? So I cannot have
>> such fields calculated...?
>>
>
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/17f167c4-ac1f-42f6-aaaa-c58594c9e8b8n%40googlegroups.com.
