Hi Joe, My aplogies, did mean to reply earlier. I think that proves it, trying to import VWS data from VWS produced text files (the so called 'Date Stamped File' according to the VWS manual) without some manual intervention is fraught with danger. The preferred option is to import from the VWS produced CSV file (the so called 'Csv File' according to the VWS manual). I don't know enough about VWS (I have not used it) to know whether the VWS generated CSV files can be generated retrospectively or not, if not then I suspect importing from VWS generated CSV files will not be much help unless the user has been producing the CSV files since they started with VWS. I suspect most users would not produce them.
Back to your case, by all means massage your data into some CSV files, you should be able to use some code to do most of the repetitive work and keep the manual changes/checks to a minimum. wee_import should take care of importing the reformatted data through its CSV import. wee_import presently only imports from a single CSV file at a time, though it should be a simple leap forward to import from multiple files (it does this for Cumulus monthly log files). I will put that down as a task to do - let me know when you are ready to import. My only advice would be to keep the day, month and year in the file name; import results are better when the records are imported chronologically - in most cases it eliminates discontinuities in rainfall across file boundaries. Gary On Friday, 2 September 2016 09:49:02 UTC+10, jmltech wrote: > > Hi Gary, > > I looked through most of the text files (towards the end I just started > randomly picking files to look at). You were correct, at least 6 different > times the format of these files changed (and then changed back). I believe > this was probably due to software updates from previous years, or me > playing with options in the VWS software without realizing that it was > affecting the data being recorded. I agree with you that trying to come up > with a program to read through each of the files and mapping correctly in > your wee_import program would not be very productive... there is no way to > account for changes made in the format, or an easy way to guess which > columns are which. I think the best solution for folks that are in a > similar situation is to create CSV file(s) that matches what your import > program is looking for. > > Once I find some time to start creating the CSV file from all of my text > files (I may write a program to somewhat automate this, stopping when it > encounters something odd or different) I'll give your latest wee_import a > try. > I appreciate your time on looking at this. I had no idea that the text > files would be inconsistent. > > Joe > > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
