On Wed, Aug 26, 2009 at 10:52:37AM -0400, David J Patrick wrote: > > Russell Adams wrote: > > David, > > > > I just noticed you'd attached a file, my bad. :P > just too subtle, I suppose :) > > > >> 1) not sure I understand the seemingly simple schema; the > >> second-to-last field is a mystery. > > > > The field list (CSVFields) in this case was taken from the header of > > the CSV files I import from my bank. Many CSV files have a similar > > header. Then each additional variable (ie: DateField) has the plain > > english text for locating that field from the header. > OK, I did figure out those fields in my csv, but I'm a bit confused > about the above paragraph.. which CVS fields file ? which header ?
All my CSV files from the bank have a header line, which gives "english" names to each column. > > This allows you to adjust how CSV2Ledger handles files with different > > input formats (ie: common fields in different orders). > ok, but that presumes a header, which my files don't got. ? > > > > I could have just made it a direct "Get date from column 1" format, > > but this seemed more intuitive. > > > >> 2) there are elements within the data set that I would like to > >> preserve as ;comments. > > > > You're in luck. By default the output from CSV2Ledger preserves the > > original CSV input as a comment with each transaction. > In retrospect, this carries a lot of functionality and data integrity, > but not yet exactly what I'm poking around for. Two comment items are created by default, one line containing the original CSV, the second containing an MD5SUM of the original CSV. There is duplicate detection on output (when enabled) which works great for when you've downloaded files with overlapping date ranges. > > > >> 3) I have a repeating set of transactions that travel in pairs and > >> are nearly identical, but have to go to different accounts > > > > CSV2Ledger handles one transaction at a time, there is no correlation > > occurring between txns. I'd suggest setting up a rule where those > > payments go into a "ToBeEdited" account so you can fix them if > > needed. > When you say "setting up a rule".. is there an existing mechanism for > that ? Maybe I'll try to have a local perl-monger make me a post-post > processor to do this, that and the other. (If I succeed, I will > > share) Setting up a rule in AccountMatching.yaml, use "ToBeEdited" instead of a real account name (ie: Expenses). > > > > There may be the later ability to say "if amount < $20 then account = > > fees else account = principal". > In this case, I think the pairing is something solid to search for. Pairing becomes arbitrarily complex, I don't think its likely that I would code it. However, you can always update the source and send me a patch. ;] > > > >> idea; if each bank spits out a "dialect" of csv, would it not be > >> good to have bank specific preprocess.yml files > > > > That may work, but given that Preprocess.yaml is typically ad-hoc > > regexp renames I don't think it would make much > > sense. FileMatches.yaml which defines the CSV header, column to field > > relationship, etc could be collected across banks. Then again, they > > are easy to make I don't think it is a problem. > Don't you think a set that was bundled for a specific institution would > be way more likely to "just work" ? No. The header/field translation is only one piece of what goes into FileMatches.yaml. Essentially. all the flags you could set on the CLI get auto-set by matching the input filename to a regexp. This updates header/field xlation, output file, image/file matching, duplicate detection, etc. Only the header/field stays common between users. > > > >> as another example, for those transactions that feature "INTERNET > >> BILL PAYMENT", I would like to turn > > > > You can rename payees using Preprocess.yaml, ie: > yes, but the detail INTERNET BILL PAYMENT should (for me) become > ; :online: > might it be possible to perform further regex/substitution passes that > have the result appended to (in this case) the payee field ? > > I could do it in post-post, but if CSV2ledger can be tweaked to do it, > that would be cleaner. CSV2Ledger internally doesn't apply any special meaning to ';', so the '; :online:' can simply be a part of your payee name. > > > > --- > > - 'm/TORONTO HYDRO ELECTRIC/' > > - 's/INTERNET BILL PMT ([0-9]+) TORONTO HYDRO ELECTRIC.*$/Toronto Hydro $1 > > ; :online:/' > what was that ?? looks like the right direction.. This is a sample Preprocessor.yaml rule. The file is yaml, and consists of arrays of regexps. > > > > Currently the check number comes from a csv column, and couldn't be > > substituted, but you can keep the number in the payee description. > currently ;-) > > > > Then you would set the account via AccountMatches.yaml, ie: > > > > --- > > - 'm/Toronto Hydro/' > > - ~ > > - 'Expenses:Utils:Hydro' > > > > The ~ on the second line indicates to use the default source account. > OK, I get that. > > > >> and if I corrected my bad syntax, and thought about it, and was a > >> real p.i.t.a, I would want; > > > > You would have to use a regexp to match each type of txn > > (offline/online/card) and sub in text for the tag. CSV2Ledger doesn't > > currently support tag writing internally, it just sees text. > right. maybe a post-post pass through a "tagger" routine, that looks > through a big long list of thing to be turned to tags.. > > > >> I don't know If you had thought about extracting some things to > >> comments, but if it can be done, it would be a boon. Again, use regexps in the preprocess stage to create your payee comments. > > > > Again, if you want a comment, just sub it in via regexp. > via CVS2ledger ? or later ? > Maybe the clean-up and tag is a job for a vim macro ! > > > >> Do we presume everything in a bank transaction has "cleared"? > >> .. maybe not.. is that another regex opportunity ? egads ! > > > > There is an execution flag to set cleared or not. I generally import > > everything as uncleared, and then when I'm done with verifying txns I > > clear them, or in bulk after I reconcile. > How do I set the flag to cleared ? CSV2Ledger --help: ... -x Mark transactions cleared > > > > Hope this helps! > Quite fantastic, and I hope I haven't been too dense. > djp > Enjoy! ------------------------------------------------------------------ Russell Adams [email protected] PGP Key ID: 0x1160DCB3 http://www.adamsinfoserv.com/ Fingerprint: 1723 D8CA 4280 1EC9 557F 66E8 1154 E018 1160 DCB3
