I have no idea why it should be 800+ mmI did the work deleting all 0.0 mm value records and left for non 0.0 days only the last record (usually either 18:00 or 22:00 whatever was available) and added them all up - result: 507.2 mm as opposite to the 470.2 mm shown in your graph and NOAA. Not 800+
PRECIPITATION (mm)
MAX ---DAYS OF RAIN---
OBS. OVER
YR MO TOTAL DAY DATE 0.30 3.00 30.00
------------------------------------------------
2023 01 21.6 17.3 05 2 2 0
2023 02 6.3 2.8 20 6 0 0
2023 03 100.8 56.4 05 7 4 2
2023 04 13.0 4.6 22 5 3 0
2023 05 111.2 56.9 31 13 7 1
2023 06 79.4 22.6 15 15 5 0
2023 07 35.9 14.7 09 12 3 0
2023 08 6.6 2.0 23 5 0 0
2023 09 68.3 48.5 25 7 3 1
2023 10 27.1 11.4 21 5 4 0
2023 11 0.0 0.0 01 0 0 0
2023 12
------------------------------------------------
470.2 56.9 May 77 31 4
also, your NOAA reports are not consistent - e.g. sum Oct 23 in the yearly
view: 27.1, in the Oct 23 monthly view: 23.9
Going through the import field map and the import CSV file,I cannot see
any reason where this (meanwhile as far as I could tell in a first rough
check) only 35 mm difference comes from.
All these consistency checks you could have done yourself and come to the same results as I did, couldn't you ? I attach the Excel file I created from your import.csv - the other columns are hidden (mark adjacent columns --> Format --> Hide/Unhide) No guarantee I missed some lines - but you can repeat the approach - and make sure all your NOAA reports are consistent amongst each other in the first place.
As a practical/pragmatic approach I suggest you complete the Excel for each month - create an additional import file with a time stamp one minute later than the last one each month (using the other data from the last record) with the missing amount and import it Or use sqlitebrowser as suggested earlier to add the missing values of the respective days to the day already imported. When you use sqlitebrowser, you will need to update the daily summaries (wee_database)
In the end, at least you will have a consistent database.If Gary or someone else has the time to analyze the data more deeply to figure out why wee_import does an incomplete job, they shall feel free to do so
On 05.11.2023 15:44, Marlon wrote:
Okie dokes. I re-exported data from ecowitt. 4 hour intervals. I cleaned up the CSVThen confirmed import conf is correct, confirmed row count ( | wc -l| and deleted my sqlite db so we can load from scratch. Import seemed fine on dry run, so I launched import. Rain total is yet again, different. This time it is 470mm (still not the 800+ it should be).Rain config set to cumulative (since ecowitt does not export yearly).Sharing the new config, csv and logs as you requested. You should be able to compare now, Rainer.Hope I can find a way to thank you and Gary for the help. On Sunday, 05 November 2023 at 02:23:37 UTC+2 Rainer Lang wrote: can you remove all lines with ";;;;;;;;;;;;;;" from the import CSV file and start the import again (I assume you still have an untouched copy of your database), re-create the NOAA reports and check again ... The NOAA reports from your web site is the only place where I could compare the import file content with your web site database. Obviously at certain days no data or not all data of that day were imported. Can you also provide the syslog from this import (or from your last import) - maybe for certain timestamps already a record existed and wee_import couldn't create this record again. wee_import only creates new records - doesn't update existing records ! On 04.11.2023 18:33, Marlon wrote:Hi guys Thanks, and apologies for a latent reply. Life got hold of me for a while. I ran the daily summaries and reweighting out of desperation, but as you mentioned, they seemed to be done auto by wee_import I am attaching the gzipped versions of the conf and csv. This particular CSV is the latest attempt I made, and has 10 min intervals (weather cloud). Previously I did the same with ecowitt (but with their varying intervals). Everything seems perfect, except rain - rain is wrong from either sites (yet displayed correctly ON those sites). I will also share the links to those if that helps to prove my sanity (or lack thereof). My WeeWX site: weewx.simple-earth.org <http://weewx.simple-earth.org> (rain wrong). My Station on Ecowitt www.ecowitt.net/home/index?id=68712 <http://www.ecowitt.net/home/index?id=68712> (rain shown correctly and matching manual meters) Thanks Gary and Rainer. Appreciated. Marlon On Thursday, 02 November 2023 at 06:59:31 UTC+2 gjr80 wrote: On Thursday, 2 November 2023 at 02:46:03 UTC+10 [email protected] wrote: 1. as Gary (@gjr80) already mentioned - send us your import.conf file for the CSV import and maybe 5 lines of your import CSV file including the first line with the descriptors No, please include the whole CSV file; if the issue is the accuracy of the totals we are going to need to see all of the source data, not just a few lines. 2. did you run the daily summary update after the import ? sudo wee_database ...... from= to= (see wee_database --help) There is no need to update the daily summaries after importing via wee_import, wee_import automatically updates the daily summaries once the data has been imported into the archive. Gary-- You received this message because you are subscribed to theGoogle Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/86c605c3-1b72-4c79-aad4-ff18bb97a7dcn%40googlegroups.com <https://groups.google.com/d/msgid/weewx-user/86c605c3-1b72-4c79-aad4-ff18bb97a7dcn%40googlegroups.com?utm_medium=email&utm_source=footer>.--You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/06c2d1f3-4972-4263-9084-085af01f890cn%40googlegroups.com <https://groups.google.com/d/msgid/weewx-user/06c2d1f3-4972-4263-9084-085af01f890cn%40googlegroups.com?utm_medium=email&utm_source=footer>.
-- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/fb4bcc42-bc82-43f5-bbd3-5907b167b7e3%40gmail.com.
weewx-import-csv-test-rain-only.xlsx
Description: MS-Excel 2007 spreadsheet
