Thanks Gary, I appreciate it. Hopefully UPDATE won't raise any serious weewx issues. I look forward to seeing it raised as an issue :)
On 18/01/2022, gjr80 <gjroder...@gmail.com> wrote: > Correct, wee_import will not accept text only fields. When we put together > wee_import we only included support for numeric fields; text fields were > omitted, perhaps not intentionally, but more likely because no real > consideration had been given to including them in the schema. That being > said, it should be a fairly simple process to modify wee_import to support > text fields, at least for CSV imports anyway. Having had a quick wee_import > > refresher this morning I am thinking we could support something like > ANComment > = ANComment, text in the field map. Let me look at this further and see > what I can come up with. > > As for supporting UPDATE, that will require further thought as I suspect > that will introduce a number of wider issues. I'll probably raise an issue > for that one. > > Gary > On Monday, 17 January 2022 at 23:14:00 UTC+10 Glenn McKechnie wrote: > >> Working with TEXT records. >> >> I've gone through the wiki entry >> https://github.com/weewx/weewx/wiki/Storing-text-in-the-database >> and that all works well. I can add TEXT to the database and display it >> in the weewx skin. So far so good. >> >> Using a modified filepile.py, I can add what I want via the new_archive >> record hook. What I can't seem to do is add historical data that is in >> TEXT form, via weewx. Fair enough, it makes sense. >> >> I figured wee_import may be a solution so I crafted a csv file and used >> wee_import but it baulks on the text field. The csv files, >> /var/tmp/weewxaddnotes.csv has the contents... >> >> timestamp,extraTemp1,extraTemp2,ANComment >> 1641729300,23.0,22.0,"Test.\\nThen test again" >> >> It appears to be wanting a weewx_unit_name and I'm missing a clue stick. >> I can't find a name that it will accept without error. If the >> weewx_unit_name field is left blank (only the csv_field_name is given) >> it gives the following. >> >> Obtaining raw import data for period 1 ... >> **** Unable to parse source-to-WeeWX field map. >> **** No units specified for source field 'ANComment' in >> /home/weewx/csv.conf. >> **** Nothing done, exiting. >> >> Which makes sense as I had the following... >> >> [[FieldMap]] >> dateTime = timestamp, unix_epoch >> interval = >> barometer = >> pressure = >> altimeter = >> inTemp = >> outTemp = >> extraTemp1 = extraTemp1, degree_C >> extraTemp2 = extraTemp2, degree_C >> ANComment = ANComment >> >> If I tell wee_imports to ignore the ANComment entry, as follows... >> ANComment = >> then everything else imports as expected so it's falling over on the >> ANComment field >> >> ignore_invalid_data in the csv.conf file is tantalizing close (a numeric >> field containing non-numeric data) but I don't want it skipped or >> halted. I want it accepted. >> >> Can I write anything into that 2nd field that will allow a TEXT entry to >> skip the conversion process, and allow it to be written as-is to the >> database? >> >> On reflection, even if I can use wee_import to add the data I want. I >> need to get it correct and complete the first time as once a RECORD is >> written (and all it's fields filled) then I can't update it later >> without going the manual route as weewx appears to be missing an inbuilt >> sql UPDATE archive method. That or I can't find it. >> >> And, that's probably so we don't shoot ourselves in the foot. >> >> But it sure would be nice to write a historical record using weewx or >> preferably wee_import (with all their built in sanity checks) and then, >> if needed later, be able to update any missing fields, or correct errant >> values without going back to the command line. >> >> I understand the risk of blindly updating existing records, but in this >> case they are all known empty or new fields, and if they do have >> something in them then it needs changing anyway! >> Caveat Emptor or something along those lines! :-) >> >> >> As an aside. I'm doing this as I have a whole host of manual rain >> records that should be entered as they are sanity checks on the >> automatic gauges, weewx seems as good a place as any to store them. >> At the same time I'll be adding extraTemp values that I've recorded >> from the haystack. >> It'd be nice to plot them, and change them, as they use existing >> fields - extraTemp1 etc. >> >> Along with that, a TEXT comment as to why some readings appear as they >> do would be useful, or where the floods were after the recent epic >> rainfall, or why the rain values appear to be stuffed, or which area of >> the haystack the probe is at. >> The latest Tongan earthquake is probably another example given the >> recent barometric burps it caused. >> >> The possibilities expand as I write. >> So I better go. >> >> But... In summary. >> >> wee_import appears to baulk on the new add TEXT method. Is there a quick >> fix? >> >> The ability to sql UPDATE any historical weewx record would be useful. 1 >> or many RECORD fields at a time. >> >> Individual confirmations not required. Other than accepting a one-time >> answer that; 'yes', I will bandage my own toes and blame no-one else if >> it backfires, because we always have our backups. >> >> Ha! Perhaps a csv.conf entry named pieces_are_all_yours with a default >> of no? Changing to 'yes' will 'turn on' the UPDATE method. ;-) >> >> >> -- >> >> Cheers >> Glenn >> >> rorpi - read only raspberry pi & various weewx addons >> https://github.com/glennmckechnie >> >> > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to weewx-user+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/75ec1eac-bd90-402d-a450-452ab9e4a960n%40googlegroups.com. > -- Cheers Glenn rorpi - read only raspberry pi & various weewx addons https://github.com/glennmckechnie -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAAraAzhS6sjB6uh3F4gL5OCWM5tOF_jnuSdznSgAX2ZWyTUogw%40mail.gmail.com.
