Correct, wee_import will not accept text only fields. When we put together wee_import we only included support for numeric fields; text fields were omitted, perhaps not intentionally, but more likely because no real consideration had been given to including them in the schema. That being said, it should be a fairly simple process to modify wee_import to support text fields, at least for CSV imports anyway. Having had a quick wee_import refresher this morning I am thinking we could support something like ANComment = ANComment, text in the field map. Let me look at this further and see what I can come up with.
As for supporting UPDATE, that will require further thought as I suspect that will introduce a number of wider issues. I'll probably raise an issue for that one. Gary On Monday, 17 January 2022 at 23:14:00 UTC+10 Glenn McKechnie wrote: > Working with TEXT records. > > I've gone through the wiki entry > https://github.com/weewx/weewx/wiki/Storing-text-in-the-database > and that all works well. I can add TEXT to the database and display it > in the weewx skin. So far so good. > > Using a modified filepile.py, I can add what I want via the new_archive > record hook. What I can't seem to do is add historical data that is in > TEXT form, via weewx. Fair enough, it makes sense. > > I figured wee_import may be a solution so I crafted a csv file and used > wee_import but it baulks on the text field. The csv files, > /var/tmp/weewxaddnotes.csv has the contents... > > timestamp,extraTemp1,extraTemp2,ANComment > 1641729300,23.0,22.0,"Test.\\nThen test again" > > It appears to be wanting a weewx_unit_name and I'm missing a clue stick. > I can't find a name that it will accept without error. If the > weewx_unit_name field is left blank (only the csv_field_name is given) > it gives the following. > > Obtaining raw import data for period 1 ... > **** Unable to parse source-to-WeeWX field map. > **** No units specified for source field 'ANComment' in > /home/weewx/csv.conf. > **** Nothing done, exiting. > > Which makes sense as I had the following... > > [[FieldMap]] > dateTime = timestamp, unix_epoch > interval = > barometer = > pressure = > altimeter = > inTemp = > outTemp = > extraTemp1 = extraTemp1, degree_C > extraTemp2 = extraTemp2, degree_C > ANComment = ANComment > > If I tell wee_imports to ignore the ANComment entry, as follows... > ANComment = > then everything else imports as expected so it's falling over on the > ANComment field > > ignore_invalid_data in the csv.conf file is tantalizing close (a numeric > field containing non-numeric data) but I don't want it skipped or > halted. I want it accepted. > > Can I write anything into that 2nd field that will allow a TEXT entry to > skip the conversion process, and allow it to be written as-is to the > database? > > On reflection, even if I can use wee_import to add the data I want. I > need to get it correct and complete the first time as once a RECORD is > written (and all it's fields filled) then I can't update it later > without going the manual route as weewx appears to be missing an inbuilt > sql UPDATE archive method. That or I can't find it. > > And, that's probably so we don't shoot ourselves in the foot. > > But it sure would be nice to write a historical record using weewx or > preferably wee_import (with all their built in sanity checks) and then, > if needed later, be able to update any missing fields, or correct errant > values without going back to the command line. > > I understand the risk of blindly updating existing records, but in this > case they are all known empty or new fields, and if they do have > something in them then it needs changing anyway! > Caveat Emptor or something along those lines! :-) > > > As an aside. I'm doing this as I have a whole host of manual rain > records that should be entered as they are sanity checks on the > automatic gauges, weewx seems as good a place as any to store them. > At the same time I'll be adding extraTemp values that I've recorded > from the haystack. > It'd be nice to plot them, and change them, as they use existing > fields - extraTemp1 etc. > > Along with that, a TEXT comment as to why some readings appear as they > do would be useful, or where the floods were after the recent epic > rainfall, or why the rain values appear to be stuffed, or which area of > the haystack the probe is at. > The latest Tongan earthquake is probably another example given the > recent barometric burps it caused. > > The possibilities expand as I write. > So I better go. > > But... In summary. > > wee_import appears to baulk on the new add TEXT method. Is there a quick > fix? > > The ability to sql UPDATE any historical weewx record would be useful. 1 > or many RECORD fields at a time. > > Individual confirmations not required. Other than accepting a one-time > answer that; 'yes', I will bandage my own toes and blame no-one else if > it backfires, because we always have our backups. > > Ha! Perhaps a csv.conf entry named pieces_are_all_yours with a default > of no? Changing to 'yes' will 'turn on' the UPDATE method. ;-) > > > -- > > Cheers > Glenn > > rorpi - read only raspberry pi & various weewx addons > https://github.com/glennmckechnie > > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/75ec1eac-bd90-402d-a450-452ab9e4a960n%40googlegroups.com.
