Re: [weewx-user] Re: Problem with wee_import utility (rain field)

[Remy Lavabre]

Thanks Gary,

My fields, including for rain, incremented automatically yesterday for the 
first time!
Your solution was the right one and I still don't understand why the 
previous time the rain was not taken into account in my weewx database

However, thank you for your explanations which helped me progress and 
allowed me to resolve my problem!

[image: Capture d’écran du 2023-09-03 14-27-35.png]



Le mercredi 30 août 2023 à 22:27:29 UTC+2, gjr80 a écrit :

> Some comments below.
>
> Gary
>
> On Wednesday, 30 August 2023 at 20:02:01 UTC+10 remy.l...@gmail.com wrote:
>
> Actually, to make it simple for you, I have a weather station that doesn't 
> have a USB output. The data is sent to AWEKAS. So in python, for 5 minutes 
> I retrieve this data regularly, make an average for certain fields 
> (temperatures, hygrometry etc.) and for others take the maximum (rain rate) 
> during these 5 minutes. For rain, the station gives me the total since 
> 00:00 which I calculate into rain over the 5 minute time slot.
> After 5 minutes, I generate a CSV which will be integrated into the weewx 
> database with Wee_import.
> Weewx recalculates all the data, generates the html page and the csv file 
> is deleted.
> We start again every 5 minutes.
>
>  
> You have a complex, automated system for using wee_import. I would be 
> looking very carefully at what data wee_import is actually being fed, it 
> might not be what you think it is.
>  
>
> Do you think that a rainRate at 0 with a rain value in the rain field 
> could cause a problem during the integration and set both fields to 0?
>
>
> I don't think this is likely. wee_import will import data if the field 
> map is valid and consistent, the field map fields are found in the source 
> data and the field value is (in this case) numeric. 0 is nothing special, 
> just another number. The import processing of one imported field has no 
> impact on any other imported field.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/4a8e03a6-e0f0-44b9-832b-572c339d04bdn%40googlegroups.com.

Re: [weewx-user] Re: Problem with wee_import utility (rain field)

[gjr80]

Some comments below.

Gary

On Wednesday, 30 August 2023 at 20:02:01 UTC+10 remy.l...@gmail.com wrote:

Actually, to make it simple for you, I have a weather station that doesn't 
have a USB output. The data is sent to AWEKAS. So in python, for 5 minutes 
I retrieve this data regularly, make an average for certain fields 
(temperatures, hygrometry etc.) and for others take the maximum (rain rate) 
during these 5 minutes. For rain, the station gives me the total since 
00:00 which I calculate into rain over the 5 minute time slot.
After 5 minutes, I generate a CSV which will be integrated into the weewx 
database with Wee_import.
Weewx recalculates all the data, generates the html page and the csv file 
is deleted.
We start again every 5 minutes.

 
You have a complex, automated system for using wee_import. I would be 
looking very carefully at what data wee_import is actually being fed, it 
might not be what you think it is.
 

Do you think that a rainRate at 0 with a rain value in the rain field could 
cause a problem during the integration and set both fields to 0?


I don't think this is likely. wee_import will import data if the field map 
is valid and consistent, the field map fields are found in the source data 
and the field value is (in this case) numeric. 0 is nothing special, just 
another number. The import processing of one imported field has no impact 
on any other imported field.

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/02fc300d-5449-4831-b6a9-2ebba3f83ce0n%40googlegroups.com.

Re: [weewx-user] Re: Problem with wee_import utility (rain field)

[Remy LAVABRE]

Thank you very much for your patience Gary,

Indeed I did not provide you with all the data from the CSV file in order
to simplify.
However, the fields were in the correct order with the correct wording.

Regarding the --dry-run command, I actually remove it for integration. This
is why all my fields are saved in the database with the timestamp (except
for rain).

Actually, to make it simple for you, I have a weather station that doesn't
have a USB output. The data is sent to AWEKAS. So in python, for 5 minutes
I retrieve this data regularly, make an average for certain fields
(temperatures, hygrometry etc.) and for others take the maximum (rain rate)
during these 5 minutes. For rain, the station gives me the total since
00:00 which I calculate into rain over the 5 minute time slot.
After 5 minutes, I generate a CSV which will be integrated into the weewx
database with Wee_import.
Weewx recalculates all the data, generates the html page and the csv file
is deleted.
We start again every 5 minutes.

The other day after the changes you told me about there was a very light
rain.
I had this rain over two 5-minute time periods with a correct rain rate in
the CSV file.
However in the weewx database nothing...

This morning, I did another test manually and put a value of 2mm in the
rain field and 4mm/h in the rainRate. I manually tried the wee_import
command (exactly the same one that is in the crontab) and I had a correct
import this time!
So the problem really comes from me and not from weewx...

Do you think that a rainRate at 0 with a rain value in the rain field could
cause a problem during the integration and set both fields to 0?

I will first try to artificially simulate a rain in python in the CSV file
and see if this value now integrates correctly or not automatically with
the crontab.
Secondly, I will monitor the actual conditions (rain is normally forecast
this weekend...) and keep you informed.
*Thank you Gary for helping me out and giving me the solution! ;-)*

*Rémy LAVABRE*


Le mar. 29 août 2023 à 22:42, gjr80  a écrit :

> I just setup a WeeWX instance with your import config file in your first
> post but using rain = discrete and a csv data file using the two records
> in your last post. I ran wee_import using these files and --dry-run and
> --verbose and received somewhat different output to what you posted, in
> my case there were warnings of missing fields in the csv data file:
>
> $ sudo /home/weewx/bin/wee_import --import-config=/var/tmp/csv.conf
> --verbose
> /home/weewx/bin/wee_import:719: DeprecationWarning: The distutils package
> is deprecated and slated for removal in Python 3.12. Use setuptools or
> check PEP 632 for potential alternatives
>   from distutils.version import StrictVersion
> Using WeeWX configuration file /home/weewx/weewx.conf
> Starting wee_import...
> A CSV import from source file '/var/tmp/remy.csv' has been requested.
> The following options will be used:
>  config=/home/weewx/weewx.conf, import-config=/var/tmp/csv.conf
>  source=/var/tmp/remy.csv, from=None, to=None
>  dry-run=None, calc_missing=True, ignore_invalid_data=True
>  tranche=250, interval=conf, date/time_string_format=%Y-%m-%d %H:%M:%S
>  delimiter=',', rain=discrete, wind_direction=[-360.0, 360.0]
>  UV=True, radiation=True
> Using database binding 'wx_binding', which is bound to database 'weewx.sdb'
> Destination table 'archive' unit system is '0x01' (US).
> Missing derived observations will be calculated.
> Starting import ...
> Obtaining raw import data for period 1 ...
> The following imported field-to-WeeWX field map will be used:
>  source field 'dateTime' in units 'unix_epoch' --> WeeWX field
> 'dateTime'
>  source field 'barometer' in units 'hPa' --> WeeWX field 'barometer'
>  source field 'inTemp' in units 'degree_C' --> WeeWX field 'inTemp'
>  source field 'outTemp' in units 'degree_C' --> WeeWX field 'outTemp'
>  source field 'inHumidity' in units 'percent' --> WeeWX field
> 'inHumidity'
>  source field 'outHumidity' in units 'percent' --> WeeWX field
> 'outHumidity'
>  source field 'windSpeed' in units 'km_per_hour' --> WeeWX field
> 'windSpeed'
>  source field 'windDir' in units 'degree_compass' --> WeeWX field
> 'windDir'
>  source field 'windGust' in units 'km_per_hour' --> WeeWX field
> 'windGust'
>  source field 'rainRate' in units 'mm_per_hour' --> WeeWX field
> 'rainRate'
>  source field 'rain' in units 'mm' --> WeeWX field 'rain'
>  source field 'dewpoint' in units 'degree_C' --> WeeWX field 'dewpoint'
>  source field 'windchill' in units 'degree_C' --> WeeWX field
> 'windchill'
>  source field 'radiation' in units 'watt_per_meter_squared' --> WeeWX
> field 'radiation'
>  source field 'UV' in units 'uv_index' --> WeeWX field 'UV'
>  source field 'extraTemp1' in units 'degree_C' --> WeeWX field
> 'extraTemp1'
> Raw import data read successfully for period 1.
> Mapping raw import data for period 

Re: [weewx-user] Re: Problem with wee_import utility (rain field)

[gjr80]

I just setup a WeeWX instance with your import config file in your first 
post but using rain = discrete and a csv data file using the two records in 
your last post. I ran wee_import using these files and --dry-run and 
--verbose and received somewhat different output to what you posted, in my 
case there were warnings of missing fields in the csv data file:

$ sudo /home/weewx/bin/wee_import --import-config=/var/tmp/csv.conf 
--verbose
/home/weewx/bin/wee_import:719: DeprecationWarning: The distutils package 
is deprecated and slated for removal in Python 3.12. Use setuptools or 
check PEP 632 for potential alternatives
  from distutils.version import StrictVersion
Using WeeWX configuration file /home/weewx/weewx.conf
Starting wee_import...
A CSV import from source file '/var/tmp/remy.csv' has been requested.
The following options will be used:
 config=/home/weewx/weewx.conf, import-config=/var/tmp/csv.conf
 source=/var/tmp/remy.csv, from=None, to=None
 dry-run=None, calc_missing=True, ignore_invalid_data=True
 tranche=250, interval=conf, date/time_string_format=%Y-%m-%d %H:%M:%S
 delimiter=',', rain=discrete, wind_direction=[-360.0, 360.0]
 UV=True, radiation=True
Using database binding 'wx_binding', which is bound to database 'weewx.sdb'
Destination table 'archive' unit system is '0x01' (US).
Missing derived observations will be calculated.
Starting import ...
Obtaining raw import data for period 1 ...
The following imported field-to-WeeWX field map will be used:
 source field 'dateTime' in units 'unix_epoch' --> WeeWX field 
'dateTime'
 source field 'barometer' in units 'hPa' --> WeeWX field 'barometer'
 source field 'inTemp' in units 'degree_C' --> WeeWX field 'inTemp'
 source field 'outTemp' in units 'degree_C' --> WeeWX field 'outTemp'
 source field 'inHumidity' in units 'percent' --> WeeWX field 
'inHumidity'
 source field 'outHumidity' in units 'percent' --> WeeWX field 
'outHumidity'
 source field 'windSpeed' in units 'km_per_hour' --> WeeWX field 
'windSpeed'
 source field 'windDir' in units 'degree_compass' --> WeeWX field 
'windDir'
 source field 'windGust' in units 'km_per_hour' --> WeeWX field 
'windGust'
 source field 'rainRate' in units 'mm_per_hour' --> WeeWX field 
'rainRate'
 source field 'rain' in units 'mm' --> WeeWX field 'rain'
 source field 'dewpoint' in units 'degree_C' --> WeeWX field 'dewpoint'
 source field 'windchill' in units 'degree_C' --> WeeWX field 
'windchill'
 source field 'radiation' in units 'watt_per_meter_squared' --> WeeWX 
field 'radiation'
 source field 'UV' in units 'uv_index' --> WeeWX field 'UV'
 source field 'extraTemp1' in units 'degree_C' --> WeeWX field 
'extraTemp1'
Raw import data read successfully for period 1.
Mapping raw import data for period 1 ...
Warning: Import field 'inTemp' is mapped to WeeWX field 'inTemp' but the
 import field 'inTemp' could not be found in one or more records.
 WeeWX field 'inTemp' will be set to 'None' in these records.
Warning: Import field 'inHumidity' is mapped to WeeWX field 'inHumidity' 
but the
 import field 'inHumidity' could not be found in one or more 
records.
 WeeWX field 'inHumidity' will be set to 'None' in these records.
Warning: Import field 'radiation' is mapped to WeeWX field 'radiation' but 
the
 import field 'radiation' could not be found in one or more records.
 WeeWX field 'radiation' will be set to 'None' in these records.
Warning: Import field 'UV' is mapped to WeeWX field 'UV' but the
 import field 'UV' could not be found in one or more records.
 WeeWX field 'UV' will be set to 'None' in these records.
Warning: Import field 'extraTemp1' is mapped to WeeWX field 'extraTemp1' 
but the
 import field 'extraTemp1' could not be found in one or more 
records.
 WeeWX field 'extraTemp1' will be set to 'None' in these records.
Mapped 2 records.
Raw import data mapped successfully for period 1.
Saving mapped data to archive for period 1 ...
2 records identified for import.
Proceeding will save all imported records in the WeeWX archive.
Are you sure you want to proceed (y/n)? y
Unique records processed: 2; Last timestamp: 2023-08-28 18:35:00 AEST 
(1693211700)
Mapped data saved to archive successfully ""for period 1.
Calculating missing derived observations ...
Processing record: 2; Last record: 2023-08-29 00:00:00 AEST (1693231200)
Recalculating daily summaries...

Finished recalculating daily summaries
Finished calculating missing derived observations
Finished import
2 records were processed and 2 unique records imported in 2.87 seconds.
Those records with a timestamp already in the archive will not have been
imported. Confirm successful import in the WeeWX log file.

These warnings are inconsequential for your import, but they make me wonder 
if you have posted the correct output for the command/config/data 
specified? Importantly, in my test 

Re: [weewx-user] Re: Problem with wee_import utility (rain field)

[Remy LAVABRE]

Hi Gary,
I followed your advice:

1/ I now have in the rain field of my csv file the amount of rain that fell
over the previous 5 minutes.
2/ I changed the config file so that rain=discrete is now the config value

Unfortunately, the integration of the csv file in the weewx database with
this new configuration did not solve the problem, the rain and rainRate
fields systematically remain at zero while the integration of all the other
fields works wonderfully!
Do you think there is something wrong or is it a weewx bug?


*My new configuration file :*
*rain = discrete*
wind_direction = 0,360

[[FieldMap]]
dateTime= dateTime, unix_epoch
barometer   = barometer, hPa
inTemp  = inTemp, degree_C
outTemp = outTemp, degree_C
inHumidity  = inHumidity, percent
outHumidity = outHumidity, percent
windSpeed   = windSpeed, km_per_hour
windDir = windDir, degree_compass
windGust= windGust, km_per_hour

*rainRate= rainRate, mm_per_hourrain= rain, mm*
dewpoint= dewpoint, degree_C
windchill   = windchill, degree_C
radiation   = radiation, watt_per_meter_squared
UV  = UV, uv_index
extraTemp1  = extraTemp1, degree_C

*Example of CSV file with rain & rainRate *:
dateTime,outTemp,outHumidity,dewpoint,appTemp,barometer,windchill,windSpeed,windDir,windGust
*,rain,rainRate*
2023-08-28 18:30:00,17.6,66.0,11.3,13.8,1009.5,17.6,3.8,0,6.1,*0.3,3.6*
2023-08-28 18:35:00,16.0,80.0,12.6,13.7,1009.4,16.0,3.8,1,7.1,*0.2,6.0*


*Result of the test :*
sudo wee_import --config=/etc/weewx/weewx.conf
--import-config=/home/pi/Documents/Bresser7in1/csv-Bresser.conf --dry-run
--verbose
Using WeeWX configuration file /etc/weewx/weewx.conf
Starting wee_import...
A CSV import from source file '/home/pi/Documents/Bresser7in1/data5.csv'
has been requested.
The following options will be used:
 config=/etc/weewx/weewx.conf,
import-config=/home/pi/Documents/Bresser7in1/csv-Bresser.conf
 source=/home/pi/Documents/Bresser7in1/data5.csv, from=None, to=None
 dry-run=True, calc_missing=True, ignore_invalid_data=True
 tranche=250, interval=conf, date/time_string_format=%Y-%m-%d %H:%M:%S
 delimiter=',', *rain=discrete,* wind_direction=[0.0, 360.0]
 UV=True, radiation=True
Using database binding 'wx_binding', which is bound to database 'weewx.sdb'
Destination table 'archive' unit system is '0x01' (US).
Missing derived observations will be calculated.
This is a dry run, imported data will not be saved to archive.
Starting dry run import ...
Obtaining raw import data for period 1 ...
The following imported field-to-WeeWX field map will be used:
 source field 'dateTime' in units 'unix_epoch' --> WeeWX field
'dateTime'
 source field 'barometer' in units 'hPa' --> WeeWX field 'barometer'
 source field 'inTemp' in units 'degree_C' --> WeeWX field 'inTemp'
 source field 'outTemp' in units 'degree_C' --> WeeWX field 'outTemp'
 source field 'inHumidity' in units 'percent' --> WeeWX field
'inHumidity'
 source field 'outHumidity' in units 'percent' --> WeeWX field
'outHumidity'
 source field 'windSpeed' in units 'km_per_hour' --> WeeWX field
'windSpeed'
 source field 'windDir' in units 'degree_compass' --> WeeWX field
'windDir'
 source field 'windGust' in units 'km_per_hour' --> WeeWX field
'windGust'


* source field 'rainRate' in units 'mm_per_hour' --> WeeWX field
'rainRate' source field 'rain' in units 'mm' --> WeeWX field 'rain'*
 source field 'dewpoint' in units 'degree_C' --> WeeWX field 'dewpoint'
 source field 'windchill' in units 'degree_C' --> WeeWX field
'windchill'
 source field 'radiation' in units 'watt_per_meter_squared' --> WeeWX
field 'radiation'
 source field 'UV' in units 'uv_index' --> WeeWX field 'UV'
 source field 'extraTemp1' in units 'degree_C' --> WeeWX field
'extraTemp1'
 source field 'extraTemp2' in units 'degree_C' --> WeeWX field
'extraTemp2'
 source field 'extraTemp3' in units 'degree_C' --> WeeWX field
'extraTemp3'
 source field 'extraTemp4' in units 'degree_C' --> WeeWX field
'extraTemp4'
 source field 'extraTemp6' in units 'degree_C' --> WeeWX field
'extraTemp6'
 source field 'extraHumid1' in units 'percent' --> WeeWX field
'extraHumid1'
 source field 'extraHumid2' in units 'percent' --> WeeWX field
'extraHumid2'
 source field 'extraHumid3' in units 'percent' --> WeeWX field
'extraHumid3'
 source field 'extraHumid4' in units 'percent' --> WeeWX field
'extraHumid4'
*Raw import data read successfully for period 1.*
Mapping raw import data for period 1 ...
Mapped 1 records.
*Raw import data mapped successfully for period 1.*
1 records identified for import.
Unique records processed: 1; Last timestamp: 2023-08-29 14:30:00 CEST
(1693312200)
Finished dry run import
1 records were processed and 1 unique records would have been imported.

*Rémy LAVABRE*

[weewx-user] Re: Problem with wee_import utility (rain field)

[Remy Lavabre]

Thank you Gary for your very quick and detailed response. Nice of you!
I had read the documentation on the Weewx utilities guide but had not 
understood it in the sense you indicate...
So I modified my Python program to calculate in the csv file the amount of 
water that fell over the 5 minute period and modified the configuration 
file with rain=discrete instead of rain=cumulative.

Thanks again Gary. I'm waiting for the next rain and confirming the 
resolution of the problem.
Unfortunately no precipitation is expected locally for the next few days... 
We'll have to be patient! ;-)
Thanks a lot again Gary

Le dimanche 27 août 2023 à 05:02:05 UTC+2, gjr80 a écrit :

> If you are importing a single record and using rain = cumulative the 
> imported rain and rainRate values will always be zero, this is expected 
> behaviour. The reason being that WeeWX records rainfall data on a per 
> archive interval basis; in other words each archive record contains the 
> rainfall that was recorded in that archive period, for example in the last 
> five minutes if using a five minute archive interval. When the import 
> source rainfall data is cumulative or a running total wee_import 
> calculates the rain field value as the difference in this cumulative or 
> running total value between successive records. The If there is only one 
> record then this difference cannot be calculated and hence rain is 
> recorded as zero. Likewise for rainRate; if there is no rain data then 
> there can be no rainRate.
>
> The use of the rain config option in a CSV import config file is covered 
> here  in the 
> Utilities 
> Guide .
>
> If you must import a single record at a time you must convert your source 
> rain data to be a per-record value and import using rain = discrete. 
> wee_import was not designed with the regular import of single records in 
> mind, rather it was designed for the bulk import of days/months/years of 
> data. 
>
> Gary
> On Sunday, 27 August 2023 at 04:34:16 UTC+10 remy.l...@gmail.com wrote:
>
>>
>>
>>
>> *Good morning,I am using the wee_import.py utility to integrate AWEKAS 
>> data into my weewx database.command executed: *sudo wee_import 
>> --config=/etc/weewx/weewx.conf 
>> --import-config=/home/pi/Documents/Bresser7in1/csv-Bresser.conf --dry-run 
>> --verbose
>>
>> *My CSVFile :*
>>
>>
>> dateTime,outTemp,outHumidity,dewpoint,appTemp,barometer,windchill,windSpeed,windDir,windGust,
>> *rain,rainRate*,UV,radiation,inTemp,inHumidity,extraTemp1
>> 2023-08-26 16:35:00,23.6,61.0,15.7,18.4,1010.6,23.6,3.2,214,3.9,*6.9,0,*
>> 1.1,101.0,26.1,53.0,24.9,22.0
>>
>>
>>
>> *My wee_import configuration file:*
>>
>> source = CSV
>>
>>
>> ##
>>
>> [CSV]
>> file = /home/pi/Documents/Bresser7in1/data5.csv
>> delimiter = ','
>> decimal = '.'
>> interval = conf
>> qc = True
>> calc_missing = True
>> ignore_invalid_data = True
>> tranche = 250
>> UV_sensor = True
>> solar_sensor = True
>> raw_datetime_format = %Y-%m-%d %H:%M:%S
>>
>> rain = cumulative
>> wind_direction = 0,360
>>
>>
>> [[FieldMap]]
>> dateTime= dateTime, unix_epoch
>> barometer   = barometer, hPa
>> inTemp  = inTemp, degree_C
>> outTemp = outTemp, degree_C
>> inHumidity  = inHumidity, percent
>> outHumidity = outHumidity, percent
>> windSpeed   = windSpeed, km_per_hour
>> windDir = windDir, degree_compass
>> windGust= windGust, km_per_hour
>> 
>> *rainRate= rainRate, mm_per_hourrain= rain, mm*
>> dewpoint= dewpoint, degree_C
>> windchill   = windchill, degree_C
>> radiation   = radiation, watt_per_meter_squared
>> UV  = UV, uv_index
>> extraTemp1  = extraTemp1, degree_C
>> extraTemp2  = extraTemp2, degree_C
>> extraTemp3  = extraTemp3, degree_C
>> extraTemp4  = extraTemp4, degree_C
>> extraTemp6  = extraTemp6, degree_C
>> extraHumid1 = extraHumid1, percent
>> extraHumid2 = extraHumid2, percent
>> extraHumid3 = extraHumid3, percent
>> extraHumid4 = extraHumid4, percent
>>
>>
>>
>> *Response from WeeWX :*
>>
>> Using WeeWX configuration file /etc/weewx/weewx.conf
>> Starting wee_import...
>> A CSV import from source file '/home/pi/Documents/Bresser7in1/data5.csv' 
>> has been requested.
>> The following options will be used:
>>  config=/etc/weewx/weewx.conf, 
>> import-config=/home/pi/Documents/Bresser7in1/csv-Bresser.conf
>>  source=/home/pi/Documents/Bresser7in1/data5.csv, from=None, to=None
>>  dry-run=True, calc_missing=True, ignore_invalid_data=True
>>  tranche=250, interval=conf, date/time_string_format=%Y-%m-%d %H:%M:%S
>>  delimiter=',', rain=cumulative, 

[weewx-user] Re: Problem with wee_import utility (rain field)

[gjr80]

If you are importing a single record and using rain = cumulative the 
imported rain and rainRate values will always be zero, this is expected 
behaviour. The reason being that WeeWX records rainfall data on a per 
archive interval basis; in other words each archive record contains the 
rainfall that was recorded in that archive period, for example in the last 
five minutes if using a five minute archive interval. When the import 
source rainfall data is cumulative or a running total wee_import calculates 
the rain field value as the difference in this cumulative or running total 
value between successive records. The If there is only one record then this 
difference cannot be calculated and hence rain is recorded as zero. 
Likewise for rainRate; if there is no rain data then there can be no 
rainRate.

The use of the rain config option in a CSV import config file is covered 
here  in the 
Utilities 
Guide .

If you must import a single record at a time you must convert your source 
rain data to be a per-record value and import using rain = discrete. 
wee_import was not designed with the regular import of single records in 
mind, rather it was designed for the bulk import of days/months/years of 
data. 

Gary
On Sunday, 27 August 2023 at 04:34:16 UTC+10 remy.l...@gmail.com wrote:

>
>
>
> *Good morning,I am using the wee_import.py utility to integrate AWEKAS 
> data into my weewx database.command executed: *sudo wee_import 
> --config=/etc/weewx/weewx.conf 
> --import-config=/home/pi/Documents/Bresser7in1/csv-Bresser.conf --dry-run 
> --verbose
>
> *My CSVFile :*
>
>
> dateTime,outTemp,outHumidity,dewpoint,appTemp,barometer,windchill,windSpeed,windDir,windGust,
> *rain,rainRate*,UV,radiation,inTemp,inHumidity,extraTemp1
> 2023-08-26 16:35:00,23.6,61.0,15.7,18.4,1010.6,23.6,3.2,214,3.9,*6.9,0,*
> 1.1,101.0,26.1,53.0,24.9,22.0
>
>
>
> *My wee_import configuration file:*
>
> source = CSV
>
>
> ##
>
> [CSV]
> file = /home/pi/Documents/Bresser7in1/data5.csv
> delimiter = ','
> decimal = '.'
> interval = conf
> qc = True
> calc_missing = True
> ignore_invalid_data = True
> tranche = 250
> UV_sensor = True
> solar_sensor = True
> raw_datetime_format = %Y-%m-%d %H:%M:%S
>
> rain = cumulative
> wind_direction = 0,360
>
>
> [[FieldMap]]
> dateTime= dateTime, unix_epoch
> barometer   = barometer, hPa
> inTemp  = inTemp, degree_C
> outTemp = outTemp, degree_C
> inHumidity  = inHumidity, percent
> outHumidity = outHumidity, percent
> windSpeed   = windSpeed, km_per_hour
> windDir = windDir, degree_compass
> windGust= windGust, km_per_hour
> 
> *rainRate= rainRate, mm_per_hourrain= rain, mm*
> dewpoint= dewpoint, degree_C
> windchill   = windchill, degree_C
> radiation   = radiation, watt_per_meter_squared
> UV  = UV, uv_index
> extraTemp1  = extraTemp1, degree_C
> extraTemp2  = extraTemp2, degree_C
> extraTemp3  = extraTemp3, degree_C
> extraTemp4  = extraTemp4, degree_C
> extraTemp6  = extraTemp6, degree_C
> extraHumid1 = extraHumid1, percent
> extraHumid2 = extraHumid2, percent
> extraHumid3 = extraHumid3, percent
> extraHumid4 = extraHumid4, percent
>
>
>
> *Response from WeeWX :*
>
> Using WeeWX configuration file /etc/weewx/weewx.conf
> Starting wee_import...
> A CSV import from source file '/home/pi/Documents/Bresser7in1/data5.csv' 
> has been requested.
> The following options will be used:
>  config=/etc/weewx/weewx.conf, 
> import-config=/home/pi/Documents/Bresser7in1/csv-Bresser.conf
>  source=/home/pi/Documents/Bresser7in1/data5.csv, from=None, to=None
>  dry-run=True, calc_missing=True, ignore_invalid_data=True
>  tranche=250, interval=conf, date/time_string_format=%Y-%m-%d %H:%M:%S
>  delimiter=',', rain=cumulative, wind_direction=[0.0, 360.0]
>  UV=True, radiation=True
> Using database binding 'wx_binding', which is bound to database 'weewx.sdb'
> Destination table 'archive' unit system is '0x01' (US).
> Missing derived observations will be calculated.
> This is a dry run, imported data will not be saved to archive.
> Starting dry run import ...
> Obtaining raw import data for period 1 ...
> The following imported field-to-WeeWX field map will be used:
>  source field 'dateTime' in units 'unix_epoch' --> WeeWX field 
> 'dateTime'
>  source field 'barometer' in units 'hPa' --> WeeWX field 'barometer'
>  source field 'inTemp' in units 'degree_C' --> WeeWX field 'inTemp'
>  source field 'outTemp' in units 'degree_C' --> WeeWX field 'outTemp'
>  source field 'inHumidity' in units 'percent' --> WeeWX