Re: [weewx-user] Re: Rain Underreading: Discrepancies even when importing from csv

['Rainer Lang' via weewx-user]

1. as Gary (@gjr80) already mentioned - send us your import.conf file 
for the CSV import and maybe 5 lines of your import CSV file including 
the first line with the descriptors

2. did you run the daily summary update after the import ?
sudo wee_database ...... from=    to= (see wee_database --help)
3. did you compare the values from the import CSV file with what was 
written to the database ?
(download/install sqlitebrowser unless already installed and start it 
with "sudo sqlitebroswer /var/lib/weewx/weewx.sdb"
(or whatever is the path to your weewx database) and compare the added 
records:
convert your date/time stamp into EPOCH to find the corresponding record 
in the first database column)

On 01.11.2023 13:40, gjr80 wrote:
As I said earlier; post or email the data you are importing along with 
the import config file. The import process is deterministic and 
relatively straightforward to troubleshoot.

Gary

On Wednesday, 1 November 2023 at 22:00:35 UTC+10 marlo...@gmail.com wrote:

    Reposting to clarify

    Thank you Rainer, most helpful.

    FWIW, I set the much longer archive, and got everything back to
    standard.
    I cannot test with rain, until it rains, but I did import data
    from weathercloud, and the final result contains the same error.
    Year total report is 401 (should be 829). Another example, October
    reports 23.9mm, but the actual number is 28.4mm

    I also previously imported from ecowitt, with the same problem.
    EcoWitt and WeatherCloud have the data correctly.

    It feels like weewx is storing rain in a way I don't understand.
    Or, the wee_import suffers from the same missing issue as normal
    interceptor.

    Edit 2: It seems almost surely that the import has the same
    erroneous result as normal operation.
    If you want to see what I mean:
    https://weewx.simple-earth.org/reports/?yr=2023 <- 401mm for the
    year vs over 800 on ecowitt (which is also what WU and my console
    and my plastic rainguage agrees on).
    I set import to cumulative, so midnight resets were valid.

    WeeWX is storing rain somewhere and forgetting about it when it
    creates graphs and reports? :D
    Sorry, I am as desperate to solve this as I am confused. Two full
    days on this now, feeling the pain.

    On Wednesday, 01 November 2023 at 10:34:11 UTC+2 Rainer Lang wrote:

        And, to improve your knowledge of your weather station, I
        recommend reading the WiKi at
        
https://www.wetterstationsforum.info/wiki/doku.php?id=wiki:wetterstationen:ecowitt-stationen
        if you understand German (the Froggit brand may point to that).
        If not, try https://www.wxforum.net/index.php?topic=40730.0 in
        English - not as extensive but the best on the web in English
        language
        by the same author

        On 01.11.2023 09:27, Rainer Lang wrote:

        I think you have to understand better how weewx works:
        even if you set a archive interval of 300 seconds (5
        minutes), each data packet sent by your console every 16
        seconds (shorter doesn't make sense for you anyway as the
        outdoor sensor array only transmits every 16 seconds) is
        processed by weewx and added to its Loop data according to
        the accumulator setup. The "loop" is a table in which all
        data which occur during an archive interval are collected
        [summarized, averaged etc.] and the last version is then
        archived at the end of the archiving interval. So nothing
        gets lost and you don't need to archive every 16 seconds ...

        You can find a proper interceptor.py and extensions.py at
        https://pc-wetterstation.de/weewx/usr/share/weewx/user/interceptor.zip
        and
        https://pc-wetterstation.de/weewx/usr/share/weewx/user/extensions.zip


        On 01.11.2023 09:17, Marlon wrote:
        Hi Rainer
        Thank you for the reply.
        You are right, that is an absurd interval (Im not sure if
        late nights or desperate times caused it).
        My console reports (well, POST's) every 16s (the minimum
        setting). I am going to increase my archive interval to 60s,
        or perhaps 32s. Not sure if it should be divisible by my
        console's abilities.

        As for the driver, thanks. Didn't know Interceptor was so
        outdated. Unfortunately, I am not sure what else my
        froggit/fineOffset WH3000 can use. I couldn't get the
        standard ones to work.

        Appreciated.
        Marlon


        On Wednesday, 01 November 2023 at 10:11:48 UTC+2 Rainer Lang
        wrote:

            a) you should use an interceptor driver version which
            cleanly handles all your sensors - and also those you
            don't have
            the original version on GitHub was never extended and is
            a crippled version from a today's point of view as it
            covers the basic sensors only and its exclusion
            field-map is incomplete - doesn't match the current
            customized server protocol sent by a modern Ecowitt
            console with more than the basic sensors connected -
            extended, complete and working versions of
            interceptor.py and extensions.py (for units) exist though
            if you use the device_type ecowitt-client as you already
            found out yourself, the assignment of the rain data
            should work properly
            But that's "beauty talk"

            b) what hits my eye is your archiving interval of 15
            seconds ...

            Nov  1 09:04:25 goatbox weewx[281767] INFO weewx.engine:
            Using archive interval of 15 seconds (software record
            generation)

            I have no idea what added value you try to get from such
            a short interval and it may overload the system and be -
            at least partially - responsible for your issue.

            On 01.11.2023 08:36, Marlon wrote:
            Sorry, I see a part of the log that was missing (i had
            it on wu-client, testing... correct setting is
            ecowitt-client in interceptor).
            Either way, the raw POST contains this:

            Nov  1 09:32:58 goatbox weewx[281767] DEBUG
            user.interceptor: POST:
            
b'PASSKEY=XXXX&stationtype=EasyWeatherV1.6.6&dateutc=2023-11-01+07:32:56&tempinf=71.1&humidityin=54&baromrelin=29.540&baromabsin=29.602&tempf=69.8&humidity=57&winddir=78&windspeedmph=1.8&windgustmph=2.2&maxdailygust=11.4&rainratein=0.000&eventrainin=0.000&hourlyrainin=0.000&dailyrainin=0.000&weeklyrainin=0.008&monthlyrainin=0.000&yearlyrainin=32.445&totalrainin=32.445&solarradiation=839.34&uv=8&wh65batt=0&freq=868M&model=WS2900_V2.01.08'

            Nov  1 09:32:58 goatbox weewx[281767] DEBUG
            user.interceptor: raw data:
            
b'PASSKEY=660D031A3A77C1332DFA76B2C975573C&stationtype=EasyWeatherV1.6.6&dateutc=2023-11-01+07:32:56&tempinf=71.1&humidityin=54&baromrelin=29.540&baromabsin=29.602&tempf=69.8&humidity=57&winddir=78&windspeedmph=1.8&windgustmph=2.2&maxdailygust=11.4&rainratein=0.000&eventrainin=0.000&hourlyrainin=0.000&dailyrainin=0.000&weeklyrainin=0.008&monthlyrainin=0.000&yearlyrainin=32.445&totalrainin=32.445&solarradiation=839.34&uv=8&wh65batt=0&freq=868M&model=WS2900_V2.01.08'

            A few types of rain params there.

            On Wednesday, 01 November 2023 at 09:08:32 UTC+2 Marlon
            wrote:

                Hey there Gary
                Your response, and the time you spent, is most
                appreciated. Thank you.

                To simplify things, let's ignore the "import"
                problem, that may (probably is) just user error and
                config. PS: for what it's worth, my rain data in
                the csv's are cumulative. So every hour, adds up
                the total until midnight where it becomes 0.0 again.

                You're triggering some ideas here, regarding
                timing. I will share my startup log head now.
                When I know the rain data matches the other sites,
                ecowitt, WU, Weathercloud... I will disable posting
                to those services and just use weewx. Even simpler.

                Under-readding seems to be at a daily level (or
                event, which is what I can humanly notice when I
                look at the console, WU, ecowitt and weewx). An
                event would end, and everything would say, eg:
                32.5mm, but weewx would be at 26mm. Even during
                rain, it would be behind the console and other
                services.

                I'm REALLY pondering the timing thing, and I think
                you're on to something about missing intercepts/timing.

                Unfortunately, I am very much in the dark about how
                these timings work. I use WSView to configure my
                FineOffset WH clone, and the minimum rate is every
                16s (which is my setting).
                With what interval and "loop packets" mean, I can't
                figure it out.

                Again, thanks for the reply.
                Startup log follows, ugly spam paste warning:

                root@goatbox:/etc/weewx# service weewx start &&
                tail -f /var/log/syslog | grep weewx
                Nov  1 09:04:25 goatbox weewx[281764] INFO
                __main__: Locale is 'C.UTF-8'
                Nov  1 09:04:25 goatbox weewx[281764] INFO
                __main__: Using configuration file
                /etc/weewx/weewx.conf
                Nov  1 09:04:25 goatbox weewx[281764] INFO
                __main__: Debug is 1
                Nov  1 09:04:25 goatbox weewx[281764] INFO
                __main__: PID file is /var/run/weewx.pid
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                __main__: Initializing engine
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.engine: Loading station type Interceptor
                (user.interceptor)
                Nov  1 09:04:25 goatbox weewx[281753]:    ...done.
                Nov  1 09:04:25 goatbox systemd[1]: Started LSB:
                weewx weather system.
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                user.interceptor: driver version is 0.60
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                user.interceptor: device type: wu-client
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                user.interceptor: hardware name: weatherstation via
                interceptor
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                user.interceptor: mode is listen
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                user.interceptor: listen on :8066
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                user.interceptor: sensor map: {'pressure':
                'pressure', 'barometer': 'barometer',
                'outHumidity': 'humidity_out', 'inHumidity':
                'humidity_in', 'outTemp': 'temperature_out',
                'inTemp': 'temperature_in', 'windSpeed':
                'wind_speed', 'windGust': 'wind_gust', 'windDir':
                'wind_dir', 'windGustDir': 'wind_gust_dir',
                'radiation': 'solar_radiation', 'dewpoint':
                'dewpoint', 'windchill': 'windchill', 'rain':
                'rain', 'rainRate': 'rain_rate', 'UV': 'uv',
                'txBatteryStatus': 'battery', 'extraTemp1':
                'temperature_1', 'extraTemp2': 'temperature_2',
                'extraTemp3': 'temperature_3', 'extraHumid1':
                'humidity_1', 'extraHumid2': 'humidity_2',
                'soilTemp1': 'soil_temperature_1', 'soilTemp2':
                'soil_temperature_2', 'soilMoist1':
                'soil_moisture_1', 'soilMoist2': 'soil_moisture_2',
                'soilMoist3': 'soil_moisture_3', 'soilMoist4':
                'soil_moisture_4', 'leafWet1': 'leafwetness_1',
                'leafWet2': 'leafwetness_2', 'pm2_5': 'pm2_5',
                'extraTemp4': 'temperature_4', 'extraTemp5':
                'temperature_5', 'extraTemp6': 'temperature_6',
                'extraTemp7': 'temperature_7', 'extraTemp8':
                'temperature_8', 'extraHumid3': 'humidity_3',
                'extraHumid4': 'humidity_4', 'extraHumid5':
                'humidity_5', 'extraHumid6': 'humidity_6',
                'extraHumid7': 'humidity_7', 'extraHumid8':
                'humidity_8', 'soilTemp3': 'soil_temperature_3',
                'soilTemp4': 'soil_temperature_4'}
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                user.interceptor: start tcp server
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service weewx.engine.StdTimeSynch
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.engine.StdTimeSynch
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service weewx.engine.StdConvert
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.engine: StdConvert target unit is 0x10
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.engine.StdConvert
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service weewx.engine.StdCalibrate
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.engine.StdCalibrate
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service weewx.engine.StdQC
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.engine.StdQC
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service
                weewx.wxservices.StdWXCalculate
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.wxservices: StdWXCalculate will use data
                binding wx_binding
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.manager: Daily summary version is 4.0
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.wxservices.StdWXCalculate
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service
                weewx.wxxtypes.StdWXXTypes
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.wxxtypes.StdWXXTypes
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service
                weewx.wxxtypes.StdPressureCooker
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.wxxtypes.StdPressureCooker
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service
                weewx.wxxtypes.StdRainRater
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.wxxtypes.StdRainRater
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service weewx.wxxtypes.StdDelta
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.wxxtypes.StdDelta
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service weewx.engine.StdArchive
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.engine: Archive will use data binding wx_binding
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.engine: Record generation will be attempted
                in 'software'
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.engine: Using archive interval of 15 seconds
                (software record generation)
                Nov  1 09:04:25 goatbox weewx[281767] WARNING
                weewx.engine: Archive delay (15) is unusually long
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Use LOOP data in hi/low calculations: 1
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.engine.StdArchive
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service
                weewx.restx.StdStationRegistry
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.restx: StationRegistry: Station will be
                registered.
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.restx.StdStationRegistry
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service
                weewx.restx.StdWunderground
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.restx: Wunderground: Posting not enabled.
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.restx.StdWunderground
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service weewx.restx.StdPWSweather
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.restx: PWSWeather: Data for station
                KLEINKAROO1 will be posted
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.restx.StdPWSweather
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service weewx.restx.StdCWOP
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.restx: CWOP: Posting not enabled.
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.restx.StdCWOP
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service weewx.restx.StdWOW
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.restx: WOW: Posting not enabled.
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.restx.StdWOW
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service weewx.restx.StdAWEKAS
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.restx: AWEKAS: Data will be uploaded for user
                marlonza
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.restx.StdAWEKAS
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service user.owm.OpenWeatherMap
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                user.owm: service version is 0.9
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                user.owm: Data will be uploaded for
                650d5a8e8885c200018ae241
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                user.owm.OpenWeatherMap
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service weewx.engine.StdPrint
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.engine.StdPrint
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Loading service weewx.engine.StdReport
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.engine: 'pyephem' detected, extended almanac
                data is available
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Finished loading service
                weewx.engine.StdReport
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                __main__: Starting up weewx version 4.10.2
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.engine: Station does not support reading the time
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.engine: Using binding 'wx_binding' to
                database 'weewx'
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.manager: Starting backfill of daily summaries
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.manager: Daily summaries up to date
                Nov  1 09:04:25 goatbox weewx[281767] INFO
                weewx.engine: Starting main packet loop.
                Nov  1 09:04:25 goatbox weewx[281767] DEBUG
                weewx.manager: Daily summary version is 4.0
                Nov  1 09:04:25 goatbox weewx[281767] message
                repeated 2 times: [ DEBUG weewx.manager: Daily
                summary version is 4.0]
                Nov  1 09:04:27 goatbox weewx[281767] DEBUG
                user.interceptor: POST:
                
b'PASSKEY=XXXX&stationtype=EasyWeatherV1.6.6&dateutc=2023-11-01+07:04:24&tempinf=69.4&humidityin=54&baromrelin=29.554&baromabsin=29.616&tempf=68.7&humidity=60&winddir=83&windspeedmph=0.2&windgustmph=1.1&maxdailygust=6.9&rainratein=0.000&eventrainin=0.000&hourlyrainin=0.000&dailyrainin=0.000&weeklyrainin=0.008&monthlyrainin=0.000&yearlyrainin=32.445&totalrainin=32.445&solarradiation=654.99&uv=6&wh65batt=0&freq=868M&model=WS2900_V2.01.08'
                Nov  1 09:04:27 goatbox weewx[281767] DEBUG
                user.interceptor: raw data:
                
b'PASSKEY=censoredC&stationtype=EasyWeatherV1.6.6&dateutc=2023-11-01+07:04:24&tempinf=69.4&humidityin=54&baromrelin=29.554&baromabsin=29.616&tempf=68.7&humidity=60&winddir=83&windspeedmph=0.2&windgustmph=1.1&maxdailygust=6.9&rainratein=0.000&eventrainin=0.000&hourlyrainin=0.000&dailyrainin=0.000&weeklyrainin=0.008&monthlyrainin=0.000&yearlyrainin=32.445&totalrainin=32.445&solarradiation=654.99&uv=6&wh65batt=0&freq=868M&model=WS2900_V2.01.08'
                Nov  1 09:04:27 goatbox weewx[281767] DEBUG
                user.interceptor: using rain_total 32.445 from
                yearlyrainin
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter
                b'PASSKEY=censored
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter
                stationtype=EasyWeatherV1.6.6
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter tempinf=69.4
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter humidityin=54
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter
                baromrelin=29.554
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter
                baromabsin=29.616
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter
                maxdailygust=6.9
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter
                rainratein=0.000
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter
                eventrainin=0.000
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter
                hourlyrainin=0.000
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter
                weeklyrainin=0.008
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter
                monthlyrainin=0.000
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter
                totalrainin=32.445
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter uv=6
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter wh65batt=0
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter freq=868M
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: unrecognized parameter
                model=WS2900_V2.01.08'
                Nov  1 09:04:27 goatbox weewx[281767] INFO
                user.interceptor: skipping rain measurement of
                32.445: no last rain
                Nov  1 09:04:27 goatbox weewx[281767] DEBUG
                user.interceptor: raw packet: {'dateTime':
                1698822264, 'usUnits': 1, 'rain_total': 32.445,
                'temperature_out': 68.7, 'humidity_out': 60.0,
                'wind_dir': 83.0, 'wind_speed': 0.2, 'wind_gust':
                1.1, 'solar_radiation': 654.99, 'rain': None}
                Nov  1 09:04:27 goatbox weewx[281767] DEBUG
                user.interceptor: mapped packet: {'dateTime':
                1698822264, 'usUnits': 1, 'outHumidity': 60.0,
                'outTemp': 68.7, 'windSpeed': 0.2, 'windGust': 1.1,
                'windDir': 83.0, 'radiation': 654.99, 'rain': None}

                Note: Rain is set to None on the first iteration.
                It reverts to 0.0 consequently.



                On Tuesday, 31 October 2023 at 11:19:18 UTC+2 gjr80
                wrote:

                    It can be tricky to maintain identical readings
                    at all destinations for stations that post to
                    multiple sources, ran in particular can be
                    tricky due to it being a (typically) cumulative
                    value whereas other observations are typically
                    averages. Small changes or the loss of a single
                    reading usually have little effect of weather
                    obs average values but loss of a mm or two of
                    rain is noticeable. Timing differences can also
                    cause discrepancies.

                    You say that 'it underreads rain by a few
                    millimeters everytime' - what does that mean,
                    is every packet received by WeeWX under or is
                    every day, week or month under? How often is
                    your station posting to ecowitt.net
                    <http://ecowitt.net> and WeeWX? It would be
                    good to see a startup log extract with debug =
                    1 so we can see your station config. If using
                    your station custom posting/upload you should
                    be seeing regular intercepts from your station
                    in the WeeWX log, are you seeing all posts from
                    your station or are some missing, do all have a
                    rain value or are some missing? Sorry but there
                    is likely going to be a lot of log analysis to
                    track this down.

                    In terms of wee_import what does your csv file
                    contain, does it contain cumulative rain value
                    or per-archive period values? What does you
                    import config file contain? There is a 'rain'
                    config option in the CSV import config file
                    <http://weewx.com/docs/4.10/utilities.htm#csv_rain>
                    that determines how rain values are handled -
                    it needs to be set correctly. if the source
                    data is correct tracking down CSV import issues
                    should be very straightforward once we have the
                    data and the config being used. Feel free to
                    post your CSV source data and import config
                    file here or email to me at gjroderick at gmail
                    dot com and I will run through them.

                    Gary
                    On Tuesday, 31 October 2023 at 17:34:53 UTC+10
                    marlo...@gmail.com wrote:

                        I nuked the DB, extracted all the data from
                        ecowitt using @comes
                        https://github.com/comes/ecowitt2weewx
                        I then imported it all back in. The data
                        from ecowitt seems correct, still. My
                        yearly rain so far is 824.1mm as per my
                        station
                        https://www.ecowitt.net/home/index?id=68712

                        After all of this, I still get about half
                        the rainfall in weewx. (490.3mm)
                        https://weewx.simple-earth.org/reports/?yr=2023

                        It's likely there is a weird field mapping,
                        or table that I just don't understand.
                        Looking at the amount of rain related
                        issues, I'm not alone. Would like to beat
                        this.
                        Please help me save my brain. It's breaking.

                        On Monday, 30 October 2023 at 13:12:46
                        UTC+2 Marlon wrote:

                            Oct 30 13:08:42 goatbox weewx[116207]
                            DEBUG weewx.reportengine: Report 'FTP'
                            not enabled. Skipping.
                            Oct 30 13:08:42 goatbox weewx[116207]
                            DEBUG weewx.reportengine: Report
                            'RSYNC' not enabled. Skipping.
                            Oct 30 13:08:49 goatbox weewx[116207]
                            DEBUG user.interceptor: empty queue
                            Oct 30 13:08:59 goatbox weewx[116207]
                            DEBUG user.interceptor: empty queue
                            Oct 30 13:09:01 goatbox weewx[116207]
                            DEBUG user.interceptor: POST:
                            
b'PASSKEY=XXXX&stationtype=EasyWeatherV1.6.6&dateutc=2023-10-30+11:08:59&tempinf=76.6&humidityin=46&baromrelin=29.661&baromabsin=29.723&tempf=71.6&humidity=44&winddir=72&windspeedmph=3.1&windgustmph=4.5&maxdailygust=15.9&rainratein=0.000&eventrainin=0.000&hourlyrainin=0.000&dailyrainin=0.000&weeklyrainin=0.008&monthlyrainin=1.118&yearlyrainin=32.445&totalrainin=32.445&solarradiation=831.15&uv=8&wh65batt=0&freq=868M&model=WS2900_V2.01.08'
                            Oct 30 13:09:01 goatbox weewx[116207]
                            DEBUG user.interceptor: raw data:
                            
b'PASSKEY=660D031A3A77C1332DFA76B2C975573C&stationtype=EasyWeatherV1.6.6&dateutc=2023-10-30+11:08:59&tempinf=76.6&humidityin=46&baromrelin=29.661&baromabsin=29.723&tempf=71.6&humidity=44&winddir=72&windspeedmph=3.1&windgustmph=4.5&maxdailygust=15.9&rainratein=0.000&eventrainin=0.000&hourlyrainin=0.000&dailyrainin=0.000&weeklyrainin=0.008&monthlyrainin=1.118&yearlyrainin=32.445&totalrainin=32.445&solarradiation=831.15&uv=8&wh65batt=0&freq=868M&model=WS2900_V2.01.08'
                            Oct 30 13:09:01 goatbox weewx[116207]
                            DEBUG user.interceptor: using
                            rain_total 32.445 from yearlyrainin
                            Oct 30 13:09:01 goatbox weewx[116207]
                            INFO user.interceptor: unrecognized
                            parameter
                            b'PASSKEY=660D031A3A77C1332DFA76B2C975573C
                            Oct 30 13:09:01 goatbox weewx[116207]
                            INFO user.interceptor: unrecognized
                            parameter stationtype=EasyWeatherV1.6.6
                            Oct 30 13:09:01 goatbox weewx[116207]
                            INFO user.interceptor: unrecognized
                            parameter baromrelin=29.661
                            Oct 30 13:09:01 goatbox weewx[116207]
                            INFO user.interceptor: unrecognized
                            parameter maxdailygust=15.9
                            Oct 30 13:09:01 goatbox weewx[116207]
                            INFO user.interceptor: unrecognized
                            parameter rainratein=0.000
                            Oct 30 13:09:01 goatbox weewx[116207]
                            INFO user.interceptor: unrecognized
                            parameter eventrainin=0.000
                            Oct 30 13:09:01 goatbox weewx[116207]
                            INFO user.interceptor: unrecognized
                            parameter hourlyrainin=0.000
                            Oct 30 13:09:01 goatbox weewx[116207]
                            DEBUG user.interceptor: ignored
                            parameter weeklyrainin=0.008
                            Oct 30 13:09:01 goatbox weewx[116207]
                            DEBUG user.interceptor: ignored
                            parameter monthlyrainin=1.118
                            Oct 30 13:09:01 goatbox weewx[116207]
                            INFO user.interceptor: unrecognized
                            parameter totalrainin=32.445
                            Oct 30 13:09:01 goatbox weewx[116207]
                            INFO user.interceptor: unrecognized
                            parameter wh65batt=0
                            Oct 30 13:09:01 goatbox weewx[116207]
                            INFO user.interceptor: unrecognized
                            parameter freq=868M
                            Oct 30 13:09:01 goatbox weewx[116207]
                            INFO user.interceptor: unrecognized
                            parameter model=WS2900_V2.01.08'
                            Oct 30 13:09:01 goatbox weewx[116207]
                            DEBUG user.interceptor: raw packet:
                            {'dateTime': 1698664139, 'usUnits': 1,
                            'rain_total': 32.445, 'temperature_in':
                            76.6, 'humidity_in': 46.0, 'pressure':
                            29.723, 'temperature_out': 71.6,
                            'humidity_out': 44.0, 'wind_dir': 72.0,
                            'wind_speed': 3.1, 'wind_gust': 4.5,
                            'solar_radiation': 831.15, 'uv': 8.0,
                            'rain': 0.0}
                            Oct 30 13:09:01 goatbox weewx[116207]
                            DEBUG user.interceptor: mapped packet:
                            {'dateTime': 1698664139, 'usUnits': 1,
                            'pressure': 29.723, 'outHumidity':
                            44.0, 'inHumidity': 46.0, 'outTemp':
                            71.6, 'inTemp': 76.6, 'windSpeed': 3.1,
                            'windGust': 4.5, 'windDir': 72.0,
                            'radiation': 831.15, 'rain': 0.0, 'UV':
                            8.0}
                            Oct 30 13:09:01 goatbox weewx[116207]
                            INFO weewx.manager: Added record
                            2023-10-30 13:08:30 SAST (1698664110)
                            to database 'weewx'
                            Oct 30 13:09:01 goatbox weewx[116207]
                            INFO weewx.manager: Added record
                            2023-10-30 13:08:30 SAST (1698664110)
                            to daily summary in 'weewx'

                            On Monday, 30 October 2023 at 12:53:37
                            UTC+2 Marlon wrote:

                                Hello
                                I recently started using weewx to
                                collect my own weather data
                                (instead of relying on external
                                sites). The problem is strange. I
                                tried for days, exhaustively, to
                                fix this - wanted to avoid another
                                support issue here that wastes your
                                time, but alas.

                                My station is a froggit WH3000 and
                                has always (still does), accurately
                                posted to ecowitt, WU and
                                WeatherCloud.

                                I added weewx on my vps now, and it
                                uses interceptor to get the data
                                using my stations 'custom' posting
                                (it has a choice of ecowitt vs wu
                                protocol).

                                The data on weewx is all fine,
                                accurate, except it underreads rain
                                by a few millimeters everytime. All
                                the data on the station console, WU
                                and ecowitt matches, but weewx
                                underreads.

                                For october for instance: 28.4mm is
                                the correct value. Weewx has 23.7mm
                                I can see it happen on daily
                                rainfall as well. Yesterday we had
                                0.2mm drizzle, but weewx still
                                thinks we had 0.0

                                On a sidenote, I tried to import
                                historic data from ecowitt (csv) to
                                weewx, and I see the same thing
                                happening.

                                I feel bested. It got the better of
                                me. Advice or ideas could go a long
                                way.
                                Thanks in advance
                                Marlon

            -- 
            You received this message because you are subscribed to
            the Google Groups "weewx-user" group.
            To unsubscribe from this group and stop receiving
            emails from it, send an email to
            weewx-user+...@googlegroups.com.
            To view this discussion on the web visit
            
https://groups.google.com/d/msgid/weewx-user/433cd173-d2f6-466e-9f16-983ef4e772e1n%40googlegroups.com
            
<https://groups.google.com/d/msgid/weewx-user/433cd173-d2f6-466e-9f16-983ef4e772e1n%40googlegroups.com?utm_medium=email&utm_source=footer>.

        -- 
        You received this message because you are subscribed to the
        Google Groups "weewx-user" group.
        To unsubscribe from this group and stop receiving emails
        from it, send an email to weewx-user+...@googlegroups.com.
        To view this discussion on the web visit
        
https://groups.google.com/d/msgid/weewx-user/55b97aba-2533-4be3-8c26-b00593da4155n%40googlegroups.com
        
<https://groups.google.com/d/msgid/weewx-user/55b97aba-2533-4be3-8c26-b00593da4155n%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
You received this message because you are subscribed to the Google 
Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/d8fc9c17-7347-4e16-8b59-620a9c28c62en%40googlegroups.com 
<https://groups.google.com/d/msgid/weewx-user/d8fc9c17-7347-4e16-8b59-620a9c28c62en%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/7c2d53db-9d35-4548-8a5e-23a3fa4de71e%40gmail.com.