I am about to import 400,000 records contained in 9 files. I did dry runs and no errors. This is 9 months of Weather-Display data in one-minute intervals. I would like to make a copy of the database before I start. I have used Sqlite before. Not sure how to use your schema to do this. Can I just copy the database and rename it? Do I need to stop weewx before copying the database? I might wait until the current weather stabilizes before stopping weewx. Thanks.
On Friday, April 21, 2017 at 10:31:05 AM UTC-4, MRL wrote: > > I am ready to do a dry run. > It took a trivial python program to convert: > day month year hour minute temperature humidity dewpoint barometer > windspeed gustspeed direction rainlastmin dailyrain monthlyrain > yearlyrain heatindex > 1 1 2017 0 1 44.1 46 24.6 29.953 0.2 0.0 245 0.000 0.000 0.000 0.000 > 44.1 > 1 1 2017 0 2 44.1 46 24.6 29.953 0.0 0.0 60 0.000 0.000 0.000 0.000 > 44.1 > 1 1 2017 0 3 44.1 46 24.6 29.953 0.3 3.0 12 0.000 0.000 0.000 0.000 > 44.1 > into: > > datetime,temperature,humidity,dewpoint,barometer,windspeed,gustspeed,direction,rainlastmin,dailyrain,monthlyrain,yearlyrain,heatindex > 2017-01-01 > 00:01:00,44.1,46,24.6,29.953,0.2,0.0,245,0.000,0.000,0.000,0.000,44.1 > 2017-01-01 > 00:02:00,44.1,46,24.6,29.953,0.0,0.0,60,0.000,0.000,0.000,0.000,44.1 > 2017-01-01 > 00:03:00,44.1,46,24.6,29.953,0.3,3.0,12,0.000,0.000,0.000,0.000,44.1 > 2017-01-01 > 00:04:00,44.1,46,24.6,29.954,0.8,2.0,12,0.000,0.000,0.000,0.000,44.1 > 2017-01-01 > 00:05:00,44.1,45,24.1,29.955,0.4,0.0,12,0.000,0.000,0.000,0.000,44.1 > > I will get back to you about the WeatherCat data as soon as the import of > the weather-view data works. > > On Friday, April 21, 2017 at 7:59:20 AM UTC-4, gjr80 wrote: >> >> On Friday, 21 April 2017 21:35:21 UTC+10, MRL wrote: >>> >>> What about the date/time? >>> The date/time is in 5 fields? Hows do I set that up in the config file? >>> >> >> If you are going to go down the path of importing from a CSV file your >> date-time data must be in one field. The field can be an epoch timestamp or >> some fixed format combination of day, month, year, hour, minute (and second >> if necessary) as long as it uniquely defines a moment in time. The format >> must be consistent throughout all rows in a given import. The way you tell >> wee_import about the format is through Python strptime() format codes. >> This is covered with examples under the heading raw_datetime_format >> <http://weewx.com/docs/utilities.htm#csv_raw_datetime_format> in the >> wee_import section of the Utilities Guide >> <http://weewx.com/docs/utilities.htm>. >> >> >>> How could the import utility deal with the strange date/time format? >>> >> >> As explained above. >> >> I would suggest you make use of the --dry-run command line option until >> you are happy with what you are importing. Either that or import into a >> test database/binding. >> >> My Weather-Cat data is in PDF files. Fortunately I have a program that >>> can convert PDF files to text files. Then a simple python program to turn >>> the output into CSV files. >>> >> >> That's too bad, I was looking forward to making wee_import WeatherCat >> aware. Are the pdfs of the monthly .cat files? If so would it be possible >> to obtain an extract from some of the pdfs, I would prefer the raw data and >> am interested in the comments I believe are at the start of each file that >> detail the format. >> >> Gary >> >> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
