George, The issue causing the error trace you posted is that you have not passed an import config file to wee_import; you have given it a weewx config file ( weewx.conf) but not an import config file which is mandatory. If your import config file is /var/tmp/csv.conf then you should use something like:
$ sudo wee_import --config=/home/weewx/weewx.conf --import-config=/var/tmp/ csv.conf If the data has been successfully imported then importing it again will not achieve much (it won't hurt either, you will just get a lot of unique constraint errors on dateTime). I think we need closer at exactly what you are seeing/monitoring and what is in your database. To start with, what files/pages are you looking at and seeing only data from 14 October onwards? is it /home/weewx/public_html/month.html? If so what are you seeing? Does the the date and time on the top of the page (assuming it is month.html) update regularly and is it correct? Is the page available via the internet or failing that can you post a screen capture? What is there and what is not could give us a clue as to what is going on. Assuming it is /home/weewx/public_html/month.html what happens when you (now) delete it and wait for it to be regenerated? Is the issue with plots on a page? if so just get into /home/weewx/public_html and delete all of the .png files, they will be regenerated on the next report cycle (some of the longer timeframe plots are only generated every few hours or once a day, deleting them forces regeneration). No stupid questions here, we were all beginners at one stage. Gary On Sunday, 25 November 2018 10:40:08 UTC+10, weerman wrote: > > > > Am Samstag, 24. November 2018 04:19:04 UTC+1 schrieb gjr80: >> >> Hi George, >> >> The errors you refer to are simply telling you that there is already a >> record with that timestamp in your archive. Take the following line: >> >> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >> record 2018-10-01 00:05:00 CEST (1538344800) to database 'weewx.sdb': >> UNIQUE constraint failed: archive.dateTime >> >> This means that there is already a record in your archive for 00:05 on 1 >> October 2018. There are two likely reasons for this; firstly if your >> previous import included data for this time then it is possible that data >> was in fact successfully imported and saved to archive. The second >> possibility is that the 00:05 1 October 2018 record was already in your >> archive before you used wee_import. If you did in fact import data for 1 >> October during your first run chances are the import successfully imported >> the data into the archive and hence the error when you try to import again. >> >> You say you deleted the month and year files from public_html, exactly >> what files are you referring to month.html and year.html or the month >> and year NOAA formatted files in public_html/NOAA ? Assuming it was the >> former, these files contain predominantly aggregates which would require >> the daily summaries. It's possible that for some reason your daily >> summaries were not properly updated during the import. Try stopping weeWX >> and rebuilding the daily summaries >> <http://weewx.com/docs/utilities.htm#Action_--rebuild-daily>. Something >> like this should do the job: >> >> $ /home/weewx/bin/wee_database --rebuild-daily >> >> Delete your month and year files again, start weeWX and let them be >> regenerated, if they still do not show the correct data then stop weeWX and >> drop <http://weewx.com/docs/utilities.htm#Action_--drop-daily> then >> rebuild the daily summaries. Again something like this: >> >> $ /home/weewx/bin/wee_database --drop-daily >> $ /home/weewx/bin/wee_database --rebuild-daily >> >> It is very unlikely to be anything to do with your station. >> >> Gary >> >> On Saturday, 24 November 2018 11:23:09 UTC+10, weerman wrote: >>> >>> Hi there, >>> >>> Currently i am trying to import my csv archive files via wee import into >>> the weewx. At first i thought everything went well, as wee import showed >>> that all the files have been processed. After that i deleted the year and >>> month files in the public_html folder and waited for the next update cycle. >>> >>> As the old data was still shown on the files, i decided to wait another >>> two hours until midnight local time. Still no change. Then i checked the >>> sql table and noticed that no new files have been added to the database at >>> all- my database still starts at October 14th 2018 7:31 PM. >>> >>> Afterwards i started a new approach by rebooting my raspberry pi >>> (thought maybe the time was wrong) and restarted weewx and the wee import. >>> This time i also made sure that syslog was running and voilá, i got the >>> following error... >>> >>> Nov 24 02:00:31 raspberrypi wee_import[2686]: wee_import: Starting >>> wee_import... >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wxcalculate: The following >>> values will be calculated: barometer=prefer_hardware, >>> windchill=prefer_hardware, dewpoint=prefer_hardware, >>> appTemp=prefer_hardware, rainRate=prefer_hardware, windrun=prefer_hardware, >>> heatindex=prefer_hardware, maxSolarRad=prefer_hardware, >>> humidex=prefer_hardware, pressure=prefer_hardware, >>> inDewpoint=prefer_hardware, ET=prefer_hardware, altimeter=prefer_hardware, >>> cloudbase=prefer_hardware >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wxcalculate: The following >>> algorithms will be used for calculations: altimeter=aaNOAA, maxSolarRad=RS >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wee_import: A CSV import >>> from source file '/home/weewx/tests/test2.csv' has been requested. >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wee_import: Using database >>> binding 'wx_binding', which is bound to database 'weewx.sdb' >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wee_import: Destination >>> table 'archive' unit system is '0x01' (US). >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wee_import: The following >>> imported field-to-weeWX field map will be used: >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wee_import: source >>> field 'Temp' in units 'degree_C' --> weeWX field 'outTemp' >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wee_import: source >>> field 'humidity' in units 'percent' --> weeWX field 'outHumidity' >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wee_import: source >>> field 'Interval' --> weeWX field 'interval' >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wee_import: source >>> field 'dayrain' in units 'mm' --> weeWX field 'rain' >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wee_import: source >>> field 'Time' --> weeWX field 'dateTime' >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wee_import: source >>> field 'wind' in units 'degree_compass' --> weeWX field 'windDir' >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wee_import: source >>> field 'pressure' in units 'hPa' --> weeWX field 'pressure' >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wee_import: source >>> field 'windspeed' in units 'km_per_hour' --> weeWX field 'windSpeed' >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wee_import: source >>> field 'gust' in units 'km_per_hour' --> weeWX field 'windGust' >>> Nov 24 02:00:32 raspberrypi wee_import[2686]: wee_import: source >>> field 'rate' in units 'mm_per_hour' --> weeWX field 'rainRate' >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 00:00:00 CEST (1538344800) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 00:05:00 CEST (1538345100) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 00:10:00 CEST (1538345400) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 00:15:00 CEST (1538345700) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 00:20:00 CEST (1538346000) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 00:25:00 CEST (1538346300) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 00:30:00 CEST (1538346600) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 00:35:00 CEST (1538346900) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 00:40:00 CEST (1538347200) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 00:45:00 CEST (1538347500) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 00:50:00 CEST (1538347800) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 00:55:00 CEST (1538348100) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 01:00:00 CEST (1538348400) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 01:05:00 CEST (1538348700) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 01:10:00 CEST (1538349000) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 01:15:00 CEST (1538349300) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 01:20:00 CEST (1538349600) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 01:25:00 CEST (1538349900) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 01:30:00 CEST (1538350200) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 01:35:00 CEST (1538350500) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 01:40:00 CEST (1538350800) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 01:45:00 CEST (1538351100) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 01:50:00 CEST (1538351400) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 01:55:00 CEST (1538351700) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 02:00:00 CEST (1538352000) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 02:05:00 CEST (1538352300) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 02:10:00 CEST (1538352600) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 02:15:00 CEST (1538352900) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 02:20:00 CEST (1538353200) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 02:25:00 CEST (1538353500) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 02:30:00 CEST (1538353800) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 02:35:00 CEST (1538354100) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 02:40:00 CEST (1538354400) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 02:45:00 CEST (1538354700) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 02:50:00 CEST (1538355000) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 02:55:00 CEST (1538355300) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> Nov 24 02:00:39 raspberrypi wee_import[2686]: manager: Unable to add >>> record 2018-10-01 03:00:00 CEST (1538355600) to database 'weewx.sdb': >>> UNIQUE constraint failed: archive.dateTime >>> >>> Anyone having an idea, what i could still try to do? My station is a >>> Fine Offset WH1080 so maybe its related to that? >>> >>> Many thanks in advance >>> >>> George >>> >> > > Hi Gary, thanks for your help. Dropped and rebuild the daily summaries and > the data already seems to be in the database (starting from January 4th > 2018). > Unfortunality the files in the public folder still show the old data > starting from October 14th onwards. On further research i tried letting > wee_import run with the config option. And with that option i got an error > message. > > pi@raspberrypi:/home/weewx/bin $ sudo wee_import > --config=/home/weewx/weewx.conf > Starting wee_import... > Traceback (most recent call last): > File "/usr/bin/wee_import", line 703, in <module> > main() > File "/usr/bin/wee_import", line 652, in main > wlog) > File "/usr/share/weewx/weeimport/weeimport.py", line 289, in > sourceFactory > file_name=options.import_config_path) > File "/usr/share/weewx/weecfg/__init__.py", line 221, in read_config > locations=locations, file_name=file_name) > File "/usr/share/weewx/weecfg/__init__.py", line 188, in find_file > candidate = os.path.abspath(os.path.join(directory, file_name)) > File "/usr/lib/python2.7/posixpath.py", line 68, in join > if b.startswith('/'): > AttributeError: 'NoneType' object has no attribute 'startswith' > > Taking another look at the documentation i found out that wee_import seems > to point to the config file path of the debian version. In fact i was using > this version when i first tried out weewx, but after a few days i switched > to the current setup python version > and deleted the weewx folder of the debian version. I am still a beginner > with linux, so please forgive my stupid question, but is it possible to > change the path to the config version somewhere- or do i have to completely > install weewx again to fix this issue? > > George > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
