Re: Getting started; assigning accounts to bank .csv data

[Simon Michael]

On 2/1/16 10:13 AM, John Hendy wrote:
> I tried to search the list for more of this sort of question, so
> forgive me if I've missed something. Replying with links pointing me
> in the right direction would be plenty sufficient if this has already
> been discussed!

Welcome John,

There are quite a lot of CSV(/OFX) to *ledger conversion tools,
including non-interactive ones; I'm not sure if you've found them all
yet. Off the top of my head, there's

- Erik Hetzner's https://gitlab.com/egh/ledger-autosync - converts OFX
not CSV, but many banks provide both. Can also download it from the bank
for you (optional).

- My own http://hledger.org/manual.html#csv - converts CSV to
ledger-compatible journal format with more options, eg you can set the
account(s) based on regexps matching the CSV records.




-- 

--- 
You received this message because you are subscribed to the Google Groups 
"Ledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ledger-cli+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Getting started; assigning accounts to bank .csv data

[Simon Michael]

Hi Stefano, I agree. Many things should be more straightforward and
better documented.

I think we could pick out a few common tasks to focus our
tool-building/documenting efforts on. Eg:

1. importing bank data and CSV generally. All of the tools and basic
generic workflows for this should be described on one page. Focus on
CSV, but we should mention OFX too (ledger-autosync is arguably best at
this with its download feature).

2. exporting all data and reports as CSV

3. moving data between the ledger-likes (ledger, hledger, beancount...).
Again, all tools and techniques gathered on one page. All existing
formats should be listed. The output of "ledger print" is a sort of
lowest common denominator, I propose we give it a name and decree that
every tool should import this as a basic interchange format. And/or a
standardised CSV representation of it, such as "hledger print -O csv"

4. moving data from and to other accounting tools (gnucash, moneydance,
excel, quick{en,books}, mobile account apps)

5. manual data entry. Editors and their modes, ledger entry, hledger add
and other prompting tools, hledger-web, recurring entry scripts, etc.

6. a catalog of journal entries covering all common transactions




On 2/4/16 12:47 AM, Stefano Zacchiroli wrote:
> I've the feeling that, right now, the lack of a generic framework ---
> generic both on the front of data source (CSV, OFX, weird bank formats,
> etc.) and on that of output formats (ledger, beancount, etc.) --- for
> semi-automatically importing transactions is perhaps the most
> significant limiting factor for the adoption of CLI accounting.
> 
> I went myself through the ad-hoc automation of my work-flow for
> importing transactions from my bank, scripting together the web outside
> of browsers suite [1] and icsv2ledger. It works decently enough for me,
> but the CLI accounting community cannot really expect every newcomer to
> go through that hacking process if it wants the community to flourish.
> 
> [1]: http://weboob.org/
> 
> Martin is right that the most front-end part of the import chain (web
> scraping in most cases) will always remain a case-by-case business. But
> there, communities like weboob can feel that niche quite nicely, if only
> they will manage to grow and be diverse enough. (Right now that
> community is very much skewed toward supporting French banks, with very
> sparse support for other international banks.)
> 
> But the rest of the toolchain, from CSV down to your favorite CLI
> accounting tool can really do better in terms of reference tools and
> automation. I'm sorry I haven't had time/energy to contribute myself to
> ledgerhub, because the design looked pretty solid; I'm looking forward
> to the new rewrite :-)
> 
> 
> 


-- 

--- 
You received this message because you are subscribed to the Google Groups 
"Ledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ledger-cli+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Getting started; assigning accounts to bank .csv data

[Stefano Zacchiroli]

On Mon, Feb 01, 2016 at 11:41:03PM -0500, Martin Blais wrote:
> Check out LedgerHub for ideas.
[...]
> The project is being killed right now, rewritten much better and simpler
> and migrated into the Beancount project; if you do end up looking at the
[...]
> In any case, you can't really get away without writing at least some
> code--it's just not realistic, the inputs from different people vary too
> much. There's very little shared code out there (just basic codes for CSV
> files, like the ones you mention) but too few users that share the same
> accounts to generate the critical mass needed for reuse. A while back I
> created the LedgerHub project to host shared importer code and provide a
> framework for doing the above, but never received much contributions and
> honestly I didn't put the care and quality attention to it I should have.

I've the feeling that, right now, the lack of a generic framework ---
generic both on the front of data source (CSV, OFX, weird bank formats,
etc.) and on that of output formats (ledger, beancount, etc.) --- for
semi-automatically importing transactions is perhaps the most
significant limiting factor for the adoption of CLI accounting.

I went myself through the ad-hoc automation of my work-flow for
importing transactions from my bank, scripting together the web outside
of browsers suite [1] and icsv2ledger. It works decently enough for me,
but the CLI accounting community cannot really expect every newcomer to
go through that hacking process if it wants the community to flourish.

[1]: http://weboob.org/

Martin is right that the most front-end part of the import chain (web
scraping in most cases) will always remain a case-by-case business. But
there, communities like weboob can feel that niche quite nicely, if only
they will manage to grow and be diverse enough. (Right now that
community is very much skewed toward supporting French banks, with very
sparse support for other international banks.)

But the rest of the toolchain, from CSV down to your favorite CLI
accounting tool can really do better in terms of reference tools and
automation. I'm sorry I haven't had time/energy to contribute myself to
ledgerhub, because the design looked pretty solid; I'm looking forward
to the new rewrite :-)


-- 
Stefano Zacchiroli  . . . . . . .  z...@upsilon.cc . . . . o . . . o . o
Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o
Former Debian Project Leader . . . . . @zacchiro . . . . o o o . . . o .
« the first rule of tautology club is the first rule of tautology club »

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"Ledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ledger-cli+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Getting started; assigning accounts to bank .csv data

[Erik Hetzner]

On Tue, 02 Feb 2016 19:48:06 -0800,
John Hendy  wrote:
> 
> 
> 
> On Monday, February 1, 2016 at 10:41:26 PM UTC-6, Martin Blais wrote:
> >
> > - Merging new transactions with previous transactions imported from the 
> > other side (e.g. a payment from a bank account to pay off on'es credit card 
> > will typically be imported from both the bank AND credit card accounts; you 
> > must merge the corresponding transactions together)
> >
> 
> Definitely. Moneydance allowed me to input an account, which would "link" 
> the transaction. Then I'd have to delete or merge the other account's 
> record of the same transaction.

One needn’t actually merge these. Here is what I do:

2015/12/31 Credit card payment
  Assets:Checking   -$100
  Transfer:AC->LCC   $100

2016/01/01 Payment received
  Liabilities:CreditCard $100
  Transfer:AC->LCC  -$100

Some kind person on the list pointed out this technique a while ago.

This makes import easier and allows for a difference in transit time. All
Transfer:* accounts should balance to $0, so you have an additional check that
everything is balancing out.

best, Erik
--
Sent from my free software system .

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"Ledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ledger-cli+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Getting started; assigning accounts to bank .csv data

[Martin Blais]

On Wed, Feb 3, 2016 at 11:30 PM, Michael Norrish 
wrote:

> One way to do two-stage cheques would be something like
>
> 2016/1/25 * My Favourite Shop
>   Expenses:Groceries   $100
>   Liabilities:Unprocessed Checks
>
> 2016/1/31 * Check clearing
>   Liabilities:Unprocessed Checks  $100
>   Assets:Checking Account
>
> You could assuredly add metadata to link the two transactions to be wrt
> some check #.
>

Yes, and I'm claiming that this linkage can probably be done by the
computer.





>
> Michael
>
> > On 4 Feb 2016, at 15:26, John Hendy  wrote:
> >
> > On Tue, Feb 2, 2016 at 10:07 PM, Martin Blais  wrote:
> >> On Tue, Feb 2, 2016 at 10:48 PM, John Hendy  wrote:
> >>>
> >>> On Monday, February 1, 2016 at 10:41:26 PM UTC-6, Martin Blais wrote:
> 
>  On Mon, Feb 1, 2016 at 1:13 PM, John Hendy  wrote:
> >
> > Greetings,
> >
> >
> > [snip]
> >
> >>>
> >>>
> >>> Huh. Yes, I'll definitely have to look into the emacs mode. I assumed
> once
> >>> it was in ledger format it would be *a lot* harder to navigate around
> vs.
> >>> just doing it while it's already in a spreadsheet format.
> >>
> >>
> >> Definitely not, text is there for your pleasure. You typically organize
> your
> >> Ledger input file in the order that makes the most sense for you (minus
> some
> >> constraints: Ledger will report the transactions in the order they
> appear in
> >> the file and the balance assertions are computed as such. Beancount
> sorts
> >> everything by date so order doesn't matter).
> >
> >
> > So, using the demo.ledger file as an example, if I run `convert` on my
> > downloaded bank file, I'm going to get something like this:
> >
> > 2010/12/01 * Checking balance
> >  Expenses:Unknown$1,000.00
> >  Equity:Unknown
> >
> > 2010/12/20 * Organic Co-op
> >  Expenses:Unknown $ 37.50  ; [=2011/01/01]
> >  Equity:Unknown   $ -225.00
> >
> > 2011/01/02 Grocery Store
> >  Expenses:Unknown $ 65.00
> >  Equity:Unknown
> >
> > Would you just go through that and manually change all of those
> > categories in ledger-mode? I still like starting from the bank .csv,
> > as it's got transaction ids and the amounts already in there... all I
> > need to do is add categories. It appears that `convert` defaults to
> > the above. As this is the primary thing of interest to me, I was sort
> > of surprised that ledger mode offered no pop-up minibuffer to edit the
> > account, at least from perusing the manual page. I only see options
> > for reconciling, reports, changing an amount, etc.
> >
> > In any case, `convert` got most of my stuff into ledger format and
> > ledger-mode at least recognizes the blocks, so I'll likely just start
> > from there. I still have *a lot* more reading to do... for example:
> >
> > - I noticed in the demo file, the co-op (which I snipped above)
> > purchases were in one chunk vs. treated as separate transactions. I
> > wouldn't default to this and am guessing it's just a preference thing
> > (compared to having one transaction per payment)?
> >
> > - I still wrestle with deposits and withdrawals. Am I the payee? Is my
> > bank? Does it matter as long as some assets category goes positive and
> > another negative?
> >
> > - I'd love tracking checks *as we write them* vs. just waiting for
> > them to appear. This used to really annoy me in Moneydance, as I'd go
> > through the checkbook once a month to see what was written but not
> > come through. Then I'd have to have these little note entries along
> > the way to remind me what the total of uncashed checks were to-date so
> > that the sums added up. I bet there was a better way in Moneydance I'd
> > missed, and I'm positive there's one in ledger/beancount.
> >
> > Anyway, still taking it slow but feel like I'm starting to get to a
> > usable noob state.
> >
> >
> > Thanks,
> > John
> >
> > --
> >
> > ---
> > You received this message because you are subscribed to the Google
> Groups "Ledger" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to ledger-cli+unsubscr...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Beancount" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to beancount+unsubscr...@googlegroups.com.
> To post to this group, send email to beanco...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/beancount/A57D9595-9C57-4D73-9276-86815B04924F%40gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"Ledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 

Re: Getting started; assigning accounts to bank .csv data

[Martin Blais]

This is related to this:

https://bitbucket.org/blais/beancount/src/tip/src/python/beancount/plugins/tag_pending.py?at=default=file-view-default

See this thread for context:
https://groups.google.com/d/msg/beancount/z9sPboW4U3c/EQk25vKcHDQJ



On Thu, Feb 4, 2016 at 1:42 AM, Martin Blais  wrote:

> On Wed, Feb 3, 2016 at 11:30 PM, Michael Norrish <
> michael.norr...@gmail.com> wrote:
>
>> One way to do two-stage cheques would be something like
>>
>> 2016/1/25 * My Favourite Shop
>>   Expenses:Groceries   $100
>>   Liabilities:Unprocessed Checks
>>
>> 2016/1/31 * Check clearing
>>   Liabilities:Unprocessed Checks  $100
>>   Assets:Checking Account
>>
>> You could assuredly add metadata to link the two transactions to be wrt
>> some check #.
>>
>
> Yes, and I'm claiming that this linkage can probably be done by the
> computer.
>
>
>
>
>
>>
>> Michael
>>
>> > On 4 Feb 2016, at 15:26, John Hendy  wrote:
>> >
>> > On Tue, Feb 2, 2016 at 10:07 PM, Martin Blais  wrote:
>> >> On Tue, Feb 2, 2016 at 10:48 PM, John Hendy 
>> wrote:
>> >>>
>> >>> On Monday, February 1, 2016 at 10:41:26 PM UTC-6, Martin Blais wrote:
>> 
>>  On Mon, Feb 1, 2016 at 1:13 PM, John Hendy 
>> wrote:
>> >
>> > Greetings,
>> >
>> >
>> > [snip]
>> >
>> >>>
>> >>>
>> >>> Huh. Yes, I'll definitely have to look into the emacs mode. I assumed
>> once
>> >>> it was in ledger format it would be *a lot* harder to navigate around
>> vs.
>> >>> just doing it while it's already in a spreadsheet format.
>> >>
>> >>
>> >> Definitely not, text is there for your pleasure. You typically
>> organize your
>> >> Ledger input file in the order that makes the most sense for you
>> (minus some
>> >> constraints: Ledger will report the transactions in the order they
>> appear in
>> >> the file and the balance assertions are computed as such. Beancount
>> sorts
>> >> everything by date so order doesn't matter).
>> >
>> >
>> > So, using the demo.ledger file as an example, if I run `convert` on my
>> > downloaded bank file, I'm going to get something like this:
>> >
>> > 2010/12/01 * Checking balance
>> >  Expenses:Unknown$1,000.00
>> >  Equity:Unknown
>> >
>> > 2010/12/20 * Organic Co-op
>> >  Expenses:Unknown $ 37.50  ; [=2011/01/01]
>> >  Equity:Unknown   $ -225.00
>> >
>> > 2011/01/02 Grocery Store
>> >  Expenses:Unknown $ 65.00
>> >  Equity:Unknown
>> >
>> > Would you just go through that and manually change all of those
>> > categories in ledger-mode? I still like starting from the bank .csv,
>> > as it's got transaction ids and the amounts already in there... all I
>> > need to do is add categories. It appears that `convert` defaults to
>> > the above. As this is the primary thing of interest to me, I was sort
>> > of surprised that ledger mode offered no pop-up minibuffer to edit the
>> > account, at least from perusing the manual page. I only see options
>> > for reconciling, reports, changing an amount, etc.
>> >
>> > In any case, `convert` got most of my stuff into ledger format and
>> > ledger-mode at least recognizes the blocks, so I'll likely just start
>> > from there. I still have *a lot* more reading to do... for example:
>> >
>> > - I noticed in the demo file, the co-op (which I snipped above)
>> > purchases were in one chunk vs. treated as separate transactions. I
>> > wouldn't default to this and am guessing it's just a preference thing
>> > (compared to having one transaction per payment)?
>> >
>> > - I still wrestle with deposits and withdrawals. Am I the payee? Is my
>> > bank? Does it matter as long as some assets category goes positive and
>> > another negative?
>> >
>> > - I'd love tracking checks *as we write them* vs. just waiting for
>> > them to appear. This used to really annoy me in Moneydance, as I'd go
>> > through the checkbook once a month to see what was written but not
>> > come through. Then I'd have to have these little note entries along
>> > the way to remind me what the total of uncashed checks were to-date so
>> > that the sums added up. I bet there was a better way in Moneydance I'd
>> > missed, and I'm positive there's one in ledger/beancount.
>> >
>> > Anyway, still taking it slow but feel like I'm starting to get to a
>> > usable noob state.
>> >
>> >
>> > Thanks,
>> > John
>> >
>> > --
>> >
>> > ---
>> > You received this message because you are subscribed to the Google
>> Groups "Ledger" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an email to ledger-cli+unsubscr...@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Beancount" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to beancount+unsubscr...@googlegroups.com.
>> To post to this 

Re: Getting started; assigning accounts to bank .csv data

[Michael Norrish]

One way to do two-stage cheques would be something like

2016/1/25 * My Favourite Shop
  Expenses:Groceries   $100
  Liabilities:Unprocessed Checks

2016/1/31 * Check clearing
  Liabilities:Unprocessed Checks  $100
  Assets:Checking Account

You could assuredly add metadata to link the two transactions to be wrt some 
check #.

Michael

> On 4 Feb 2016, at 15:26, John Hendy  wrote:
> 
> On Tue, Feb 2, 2016 at 10:07 PM, Martin Blais  wrote:
>> On Tue, Feb 2, 2016 at 10:48 PM, John Hendy  wrote:
>>> 
>>> On Monday, February 1, 2016 at 10:41:26 PM UTC-6, Martin Blais wrote:
 
 On Mon, Feb 1, 2016 at 1:13 PM, John Hendy  wrote:
> 
> Greetings,
> 
> 
> [snip]
> 
>>> 
>>> 
>>> Huh. Yes, I'll definitely have to look into the emacs mode. I assumed once
>>> it was in ledger format it would be *a lot* harder to navigate around vs.
>>> just doing it while it's already in a spreadsheet format.
>> 
>> 
>> Definitely not, text is there for your pleasure. You typically organize your
>> Ledger input file in the order that makes the most sense for you (minus some
>> constraints: Ledger will report the transactions in the order they appear in
>> the file and the balance assertions are computed as such. Beancount sorts
>> everything by date so order doesn't matter).
> 
> 
> So, using the demo.ledger file as an example, if I run `convert` on my
> downloaded bank file, I'm going to get something like this:
> 
> 2010/12/01 * Checking balance
>  Expenses:Unknown$1,000.00
>  Equity:Unknown
> 
> 2010/12/20 * Organic Co-op
>  Expenses:Unknown $ 37.50  ; [=2011/01/01]
>  Equity:Unknown   $ -225.00
> 
> 2011/01/02 Grocery Store
>  Expenses:Unknown $ 65.00
>  Equity:Unknown
> 
> Would you just go through that and manually change all of those
> categories in ledger-mode? I still like starting from the bank .csv,
> as it's got transaction ids and the amounts already in there... all I
> need to do is add categories. It appears that `convert` defaults to
> the above. As this is the primary thing of interest to me, I was sort
> of surprised that ledger mode offered no pop-up minibuffer to edit the
> account, at least from perusing the manual page. I only see options
> for reconciling, reports, changing an amount, etc.
> 
> In any case, `convert` got most of my stuff into ledger format and
> ledger-mode at least recognizes the blocks, so I'll likely just start
> from there. I still have *a lot* more reading to do... for example:
> 
> - I noticed in the demo file, the co-op (which I snipped above)
> purchases were in one chunk vs. treated as separate transactions. I
> wouldn't default to this and am guessing it's just a preference thing
> (compared to having one transaction per payment)?
> 
> - I still wrestle with deposits and withdrawals. Am I the payee? Is my
> bank? Does it matter as long as some assets category goes positive and
> another negative?
> 
> - I'd love tracking checks *as we write them* vs. just waiting for
> them to appear. This used to really annoy me in Moneydance, as I'd go
> through the checkbook once a month to see what was written but not
> come through. Then I'd have to have these little note entries along
> the way to remind me what the total of uncashed checks were to-date so
> that the sums added up. I bet there was a better way in Moneydance I'd
> missed, and I'm positive there's one in ledger/beancount.
> 
> Anyway, still taking it slow but feel like I'm starting to get to a
> usable noob state.
> 
> 
> Thanks,
> John
> 
> -- 
> 
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Ledger" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to ledger-cli+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"Ledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ledger-cli+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Getting started; assigning accounts to bank .csv data

[David Glasser]

One downside to doing it this way is that before you enter the check
clearing transaction, Assets:Checking does not actually answer the question
"how much can I take out of my checking account without bouncing a check",
which surely is a very important use case.

--dave
On Feb 3, 2016 8:30 PM, "Michael Norrish"  wrote:

> One way to do two-stage cheques would be something like
>
> 2016/1/25 * My Favourite Shop
>   Expenses:Groceries   $100
>   Liabilities:Unprocessed Checks
>
> 2016/1/31 * Check clearing
>   Liabilities:Unprocessed Checks  $100
>   Assets:Checking Account
>
> You could assuredly add metadata to link the two transactions to be wrt
> some check #.
>
> Michael
>
> > On 4 Feb 2016, at 15:26, John Hendy  wrote:
> >
> > On Tue, Feb 2, 2016 at 10:07 PM, Martin Blais  wrote:
> >> On Tue, Feb 2, 2016 at 10:48 PM, John Hendy  wrote:
> >>>
> >>> On Monday, February 1, 2016 at 10:41:26 PM UTC-6, Martin Blais wrote:
> 
>  On Mon, Feb 1, 2016 at 1:13 PM, John Hendy  wrote:
> >
> > Greetings,
> >
> >
> > [snip]
> >
> >>>
> >>>
> >>> Huh. Yes, I'll definitely have to look into the emacs mode. I assumed
> once
> >>> it was in ledger format it would be *a lot* harder to navigate around
> vs.
> >>> just doing it while it's already in a spreadsheet format.
> >>
> >>
> >> Definitely not, text is there for your pleasure. You typically organize
> your
> >> Ledger input file in the order that makes the most sense for you (minus
> some
> >> constraints: Ledger will report the transactions in the order they
> appear in
> >> the file and the balance assertions are computed as such. Beancount
> sorts
> >> everything by date so order doesn't matter).
> >
> >
> > So, using the demo.ledger file as an example, if I run `convert` on my
> > downloaded bank file, I'm going to get something like this:
> >
> > 2010/12/01 * Checking balance
> >  Expenses:Unknown$1,000.00
> >  Equity:Unknown
> >
> > 2010/12/20 * Organic Co-op
> >  Expenses:Unknown $ 37.50  ; [=2011/01/01]
> >  Equity:Unknown   $ -225.00
> >
> > 2011/01/02 Grocery Store
> >  Expenses:Unknown $ 65.00
> >  Equity:Unknown
> >
> > Would you just go through that and manually change all of those
> > categories in ledger-mode? I still like starting from the bank .csv,
> > as it's got transaction ids and the amounts already in there... all I
> > need to do is add categories. It appears that `convert` defaults to
> > the above. As this is the primary thing of interest to me, I was sort
> > of surprised that ledger mode offered no pop-up minibuffer to edit the
> > account, at least from perusing the manual page. I only see options
> > for reconciling, reports, changing an amount, etc.
> >
> > In any case, `convert` got most of my stuff into ledger format and
> > ledger-mode at least recognizes the blocks, so I'll likely just start
> > from there. I still have *a lot* more reading to do... for example:
> >
> > - I noticed in the demo file, the co-op (which I snipped above)
> > purchases were in one chunk vs. treated as separate transactions. I
> > wouldn't default to this and am guessing it's just a preference thing
> > (compared to having one transaction per payment)?
> >
> > - I still wrestle with deposits and withdrawals. Am I the payee? Is my
> > bank? Does it matter as long as some assets category goes positive and
> > another negative?
> >
> > - I'd love tracking checks *as we write them* vs. just waiting for
> > them to appear. This used to really annoy me in Moneydance, as I'd go
> > through the checkbook once a month to see what was written but not
> > come through. Then I'd have to have these little note entries along
> > the way to remind me what the total of uncashed checks were to-date so
> > that the sums added up. I bet there was a better way in Moneydance I'd
> > missed, and I'm positive there's one in ledger/beancount.
> >
> > Anyway, still taking it slow but feel like I'm starting to get to a
> > usable noob state.
> >
> >
> > Thanks,
> > John
> >
> > --
> >
> > ---
> > You received this message because you are subscribed to the Google
> Groups "Ledger" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to ledger-cli+unsubscr...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "Ledger" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ledger-cli+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"Ledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ledger-cli+unsubscr...@googlegroups.com.

Re: Getting started; assigning accounts to bank .csv data

[Michael Norrish]

A balance command mentioning those two accounts would sum to the amount you 
want:

   ledger bal assets.checking liabilities:unprocessed

Michael

> On 4 Feb 2016, at 15:50, David Glasser  wrote:
> 
> One downside to doing it this way is that before you enter the check
> clearing transaction, Assets:Checking does not actually answer the question
> "how much can I take out of my checking account without bouncing a check",
> which surely is a very important use case.
> 
> --dave
> 
> On Feb 3, 2016 8:30 PM, "Michael Norrish"  > wrote:
> One way to do two-stage cheques would be something like
> 
> 2016/1/25 * My Favourite Shop
>   Expenses:Groceries   $100
>   Liabilities:Unprocessed Checks
> 
> 2016/1/31 * Check clearing
>   Liabilities:Unprocessed Checks  $100
>   Assets:Checking Account
> 
> You could assuredly add metadata to link the two transactions to be wrt some 
> check #.
> 
> Michael
> 
> > On 4 Feb 2016, at 15:26, John Hendy  > > wrote:
> >
> > On Tue, Feb 2, 2016 at 10:07 PM, Martin Blais  > > wrote:
> >> On Tue, Feb 2, 2016 at 10:48 PM, John Hendy  >> > wrote:
> >>>
> >>> On Monday, February 1, 2016 at 10:41:26 PM UTC-6, Martin Blais wrote:
> 
>  On Mon, Feb 1, 2016 at 1:13 PM, John Hendy   > wrote:
> >
> > Greetings,
> >
> >
> > [snip]
> >
> >>>
> >>>
> >>> Huh. Yes, I'll definitely have to look into the emacs mode. I assumed once
> >>> it was in ledger format it would be *a lot* harder to navigate around vs.
> >>> just doing it while it's already in a spreadsheet format.
> >>
> >>
> >> Definitely not, text is there for your pleasure. You typically organize 
> >> your
> >> Ledger input file in the order that makes the most sense for you (minus 
> >> some
> >> constraints: Ledger will report the transactions in the order they appear 
> >> in
> >> the file and the balance assertions are computed as such. Beancount sorts
> >> everything by date so order doesn't matter).
> >
> >
> > So, using the demo.ledger file as an example, if I run `convert` on my
> > downloaded bank file, I'm going to get something like this:
> >
> > 2010/12/01 * Checking balance
> >  Expenses:Unknown$1,000.00
> >  Equity:Unknown
> >
> > 2010/12/20 * Organic Co-op
> >  Expenses:Unknown $ 37.50  ; [=2011/01/01]
> >  Equity:Unknown   $ -225.00
> >
> > 2011/01/02 Grocery Store
> >  Expenses:Unknown $ 65.00
> >  Equity:Unknown
> >
> > Would you just go through that and manually change all of those
> > categories in ledger-mode? I still like starting from the bank .csv,
> > as it's got transaction ids and the amounts already in there... all I
> > need to do is add categories. It appears that `convert` defaults to
> > the above. As this is the primary thing of interest to me, I was sort
> > of surprised that ledger mode offered no pop-up minibuffer to edit the
> > account, at least from perusing the manual page. I only see options
> > for reconciling, reports, changing an amount, etc.
> >
> > In any case, `convert` got most of my stuff into ledger format and
> > ledger-mode at least recognizes the blocks, so I'll likely just start
> > from there. I still have *a lot* more reading to do... for example:
> >
> > - I noticed in the demo file, the co-op (which I snipped above)
> > purchases were in one chunk vs. treated as separate transactions. I
> > wouldn't default to this and am guessing it's just a preference thing
> > (compared to having one transaction per payment)?
> >
> > - I still wrestle with deposits and withdrawals. Am I the payee? Is my
> > bank? Does it matter as long as some assets category goes positive and
> > another negative?
> >
> > - I'd love tracking checks *as we write them* vs. just waiting for
> > them to appear. This used to really annoy me in Moneydance, as I'd go
> > through the checkbook once a month to see what was written but not
> > come through. Then I'd have to have these little note entries along
> > the way to remind me what the total of uncashed checks were to-date so
> > that the sums added up. I bet there was a better way in Moneydance I'd
> > missed, and I'm positive there's one in ledger/beancount.
> >
> > Anyway, still taking it slow but feel like I'm starting to get to a
> > usable noob state.
> >
> >
> > Thanks,
> > John
> >
> > --
> >
> > ---
> > You received this message because you are subscribed to the Google Groups 
> > "Ledger" group.
> > To unsubscribe from this group and stop receiving emails from it, send an 
> > email to ledger-cli+unsubscr...@googlegroups.com 
> > .
> > For more options, visit https://groups.google.com/d/optout 
> > .
> 

Re: Getting started; assigning accounts to bank .csv data

[John Hendy]

On Tue, Feb 2, 2016 at 10:07 PM, Martin Blais  wrote:
> On Tue, Feb 2, 2016 at 10:48 PM, John Hendy  wrote:
>>
>> On Monday, February 1, 2016 at 10:41:26 PM UTC-6, Martin Blais wrote:
>>>
>>> On Mon, Feb 1, 2016 at 1:13 PM, John Hendy  wrote:

 Greetings,


[snip]

>>
>>
>> Huh. Yes, I'll definitely have to look into the emacs mode. I assumed once
>> it was in ledger format it would be *a lot* harder to navigate around vs.
>> just doing it while it's already in a spreadsheet format.
>
>
> Definitely not, text is there for your pleasure. You typically organize your
> Ledger input file in the order that makes the most sense for you (minus some
> constraints: Ledger will report the transactions in the order they appear in
> the file and the balance assertions are computed as such. Beancount sorts
> everything by date so order doesn't matter).


So, using the demo.ledger file as an example, if I run `convert` on my
downloaded bank file, I'm going to get something like this:

2010/12/01 * Checking balance
  Expenses:Unknown$1,000.00
  Equity:Unknown

2010/12/20 * Organic Co-op
  Expenses:Unknown $ 37.50  ; [=2011/01/01]
  Equity:Unknown   $ -225.00

2011/01/02 Grocery Store
  Expenses:Unknown $ 65.00
  Equity:Unknown

Would you just go through that and manually change all of those
categories in ledger-mode? I still like starting from the bank .csv,
as it's got transaction ids and the amounts already in there... all I
need to do is add categories. It appears that `convert` defaults to
the above. As this is the primary thing of interest to me, I was sort
of surprised that ledger mode offered no pop-up minibuffer to edit the
account, at least from perusing the manual page. I only see options
for reconciling, reports, changing an amount, etc.

In any case, `convert` got most of my stuff into ledger format and
ledger-mode at least recognizes the blocks, so I'll likely just start
from there. I still have *a lot* more reading to do... for example:

- I noticed in the demo file, the co-op (which I snipped above)
purchases were in one chunk vs. treated as separate transactions. I
wouldn't default to this and am guessing it's just a preference thing
(compared to having one transaction per payment)?

- I still wrestle with deposits and withdrawals. Am I the payee? Is my
bank? Does it matter as long as some assets category goes positive and
another negative?

- I'd love tracking checks *as we write them* vs. just waiting for
them to appear. This used to really annoy me in Moneydance, as I'd go
through the checkbook once a month to see what was written but not
come through. Then I'd have to have these little note entries along
the way to remind me what the total of uncashed checks were to-date so
that the sums added up. I bet there was a better way in Moneydance I'd
missed, and I'm positive there's one in ledger/beancount.

Anyway, still taking it slow but feel like I'm starting to get to a
usable noob state.


Thanks,
John

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"Ledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ledger-cli+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Getting started; assigning accounts to bank .csv data

[David Glasser]

Ah, sure, if there's a 1:1 mapping between unprocessed accounts and
main accounts.  (Perhaps Assets:Checking:Unprocessed Checks?)

On Wed, Feb 3, 2016 at 9:32 PM, Michael Norrish
 wrote:
> A balance command mentioning those two accounts would sum to the amount you
> want:
>
>ledger bal assets.checking liabilities:unprocessed
>
> Michael
>
> On 4 Feb 2016, at 15:50, David Glasser  wrote:
>
> One downside to doing it this way is that before you enter the check
> clearing transaction, Assets:Checking does not actually answer the question
> "how much can I take out of my checking account without bouncing a check",
> which surely is a very important use case.
>
> --dave
>
> On Feb 3, 2016 8:30 PM, "Michael Norrish"  wrote:
>>
>> One way to do two-stage cheques would be something like
>>
>> 2016/1/25 * My Favourite Shop
>>   Expenses:Groceries   $100
>>   Liabilities:Unprocessed Checks
>>
>> 2016/1/31 * Check clearing
>>   Liabilities:Unprocessed Checks  $100
>>   Assets:Checking Account
>>
>> You could assuredly add metadata to link the two transactions to be wrt
>> some check #.
>>
>> Michael
>>
>> > On 4 Feb 2016, at 15:26, John Hendy  wrote:
>> >
>> > On Tue, Feb 2, 2016 at 10:07 PM, Martin Blais  wrote:
>> >> On Tue, Feb 2, 2016 at 10:48 PM, John Hendy  wrote:
>> >>>
>> >>> On Monday, February 1, 2016 at 10:41:26 PM UTC-6, Martin Blais wrote:
>> 
>>  On Mon, Feb 1, 2016 at 1:13 PM, John Hendy  wrote:
>> >
>> > Greetings,
>> >
>> >
>> > [snip]
>> >
>> >>>
>> >>>
>> >>> Huh. Yes, I'll definitely have to look into the emacs mode. I assumed
>> >>> once
>> >>> it was in ledger format it would be *a lot* harder to navigate around
>> >>> vs.
>> >>> just doing it while it's already in a spreadsheet format.
>> >>
>> >>
>> >> Definitely not, text is there for your pleasure. You typically organize
>> >> your
>> >> Ledger input file in the order that makes the most sense for you (minus
>> >> some
>> >> constraints: Ledger will report the transactions in the order they
>> >> appear in
>> >> the file and the balance assertions are computed as such. Beancount
>> >> sorts
>> >> everything by date so order doesn't matter).
>> >
>> >
>> > So, using the demo.ledger file as an example, if I run `convert` on my
>> > downloaded bank file, I'm going to get something like this:
>> >
>> > 2010/12/01 * Checking balance
>> >  Expenses:Unknown$1,000.00
>> >  Equity:Unknown
>> >
>> > 2010/12/20 * Organic Co-op
>> >  Expenses:Unknown $ 37.50  ; [=2011/01/01]
>> >  Equity:Unknown   $ -225.00
>> >
>> > 2011/01/02 Grocery Store
>> >  Expenses:Unknown $ 65.00
>> >  Equity:Unknown
>> >
>> > Would you just go through that and manually change all of those
>> > categories in ledger-mode? I still like starting from the bank .csv,
>> > as it's got transaction ids and the amounts already in there... all I
>> > need to do is add categories. It appears that `convert` defaults to
>> > the above. As this is the primary thing of interest to me, I was sort
>> > of surprised that ledger mode offered no pop-up minibuffer to edit the
>> > account, at least from perusing the manual page. I only see options
>> > for reconciling, reports, changing an amount, etc.
>> >
>> > In any case, `convert` got most of my stuff into ledger format and
>> > ledger-mode at least recognizes the blocks, so I'll likely just start
>> > from there. I still have *a lot* more reading to do... for example:
>> >
>> > - I noticed in the demo file, the co-op (which I snipped above)
>> > purchases were in one chunk vs. treated as separate transactions. I
>> > wouldn't default to this and am guessing it's just a preference thing
>> > (compared to having one transaction per payment)?
>> >
>> > - I still wrestle with deposits and withdrawals. Am I the payee? Is my
>> > bank? Does it matter as long as some assets category goes positive and
>> > another negative?
>> >
>> > - I'd love tracking checks *as we write them* vs. just waiting for
>> > them to appear. This used to really annoy me in Moneydance, as I'd go
>> > through the checkbook once a month to see what was written but not
>> > come through. Then I'd have to have these little note entries along
>> > the way to remind me what the total of uncashed checks were to-date so
>> > that the sums added up. I bet there was a better way in Moneydance I'd
>> > missed, and I'm positive there's one in ledger/beancount.
>> >
>> > Anyway, still taking it slow but feel like I'm starting to get to a
>> > usable noob state.
>> >
>> >
>> > Thanks,
>> > John
>> >
>> > --
>> >
>> > ---
>> > You received this message because you are subscribed to the Google
>> > Groups "Ledger" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an email to 

Re: Getting started; assigning accounts to bank .csv data

[Martin Blais]

On Wed, Feb 3, 2016 at 11:48 AM, Erik Hetzner  wrote:

> On Tue, 02 Feb 2016 19:48:06 -0800,
> John Hendy  wrote:
> >
> >
> >
> > On Monday, February 1, 2016 at 10:41:26 PM UTC-6, Martin Blais wrote:
> > >
> > > - Merging new transactions with previous transactions imported from the
> > > other side (e.g. a payment from a bank account to pay off on'es credit
> card
> > > will typically be imported from both the bank AND credit card
> accounts; you
> > > must merge the corresponding transactions together)
> > >
> >
> > Definitely. Moneydance allowed me to input an account, which would "link"
> > the transaction. Then I'd have to delete or merge the other account's
> > record of the same transaction.
>
> One needn’t actually merge these. Here is what I do:
>
> 2015/12/31 Credit card payment
>   Assets:Checking   -$100
>   Transfer:AC->LCC   $100
>
> 2016/01/01 Payment received
>   Liabilities:CreditCard $100
>   Transfer:AC->LCC  -$100
>
> Some kind person on the list pointed out this technique a while ago.
>
> This makes import easier and allows for a difference in transit time. All
> Transfer:* accounts should balance to $0, so you have an additional check
> that
> everything is balancing out.
>

I think it would be possible to process the stream of transactions and
identify close matches based on common accounts and nearby dates, and
automatically merge matching transactions into a single one, removing zero
balances. The dual operation is assigning individual dates on postings of a
single transaction and having the software split them up to obtain the
result you describe.

In either case, matching transactions should be linked automatically and it
should be possible to report on a set of matching transactions (a-la
"bean-doctor linked").

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"Ledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ledger-cli+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Getting started; assigning accounts to bank .csv data

[Martin Blais]

BTW, here's an auto-generated example file that looks similar to how I
organize mine using org-mode:
https://bitbucket.org/blais/beancount/src/tip/examples/example.beancount


On Tue, Feb 2, 2016 at 11:07 PM, Martin Blais  wrote:

> On Tue, Feb 2, 2016 at 10:48 PM, John Hendy  wrote:
>>
>> On Monday, February 1, 2016 at 10:41:26 PM UTC-6, Martin Blais wrote:
>>
>>> On Mon, Feb 1, 2016 at 1:13 PM, John Hendy  wrote:
>>>
 Greetings,


 It's a fresh year and I've been seeing ledger come up on the Org-mode
 mailing list for some time and decided to give it a try. I'm coming
 from Moneydance and just wanted to get away from the tedious GUI
 method of adding information, as well as have flexibility to generate
 my own reports/visualizations with python or R, etc. [1]

 Consider that I'm about a week into reading through docs here and
 there during evenings. My first step was going to be importing a
 downloaded .csv from my bank to get started. I'm still trying to
 verify I get the terminology, so I'll use this from the manual:

 From 5.1 Basic format:
 ```
 This transaction has a date, a payee or description, a target account
 (the first posting), and a source account (the second posting). Each
 posting specifies what action is taken related to that account.
 ```

 From 7.2.1.2 The convert command:
 ```
 The fields ledger can recognize contain these case-insensitive strings
 date, posted, code, payee or desc or description, amount, cost,total,
 and note.
 ```

 For my purposes, I import my finances primarily to "categorize" (what
 I believe here is called adding an account) and assign a payee so that
 I can track my spending against a budget. So, I'm surprised there's no
 special column keyword I can add for "account". It appears that all I
 can do is pass, say, `--account "assets:checking"` to have ledger know
 it's against assets:checking? Is that correct?

 From trying to google "import csv account ledger" or similar
 variations, I've been surprised that the only tools to do something
 like this appear to be interactive one-trans-at-a-time programs like
 icsv2ledger and reckon (granted, they can learn or follow rules). I
 could quickly go through my bank's .csv and add exp:food:dining,
 exp:auto:fuel to my ~100 transactions a month and have those imported
 just like the other column data.

>>>
>>>
>> Thanks for the awesome reply!
>>
>>
>>> Keep in mind that part of the process of importing (they like call it
>>> "reconciling") involves
>>> - Manually reviewing the transactions for correctness or fraud
>>>
>>
>> I'll get there. For better or worse, I take the downloaded bank .csv as
>> "truth" and am mostly interested in getting a better handle on what my
>> money is used on, budgeting, planning, etc.
>>
>>
>>> - Merging new transactions with previous transactions imported from the
>>> other side (e.g. a payment from a bank account to pay off on'es credit card
>>> will typically be imported from both the bank AND credit card accounts; you
>>> must merge the corresponding transactions together)
>>>
>>
>> Definitely. Moneydance allowed me to input an account, which would "link"
>> the transaction. Then I'd have to delete or merge the other account's
>> record of the same transaction.
>>
>
> BTW, there are some ideas around about automatically merging two
> incomplete transactions. This problem is the dual of solving the issue of
> settlement dates, i.e., the problem being that the dates of each of the two
> sides may settlement on different days.
> See http://furius.ca/beancount/doc/proposal-settlement for some
> ruminations and scour the mailing-list, there is more discussion about this.
>
>
>
> - Assigning the right category (you can automate this with a script I
>>> suppose; frankly it's not much work, I do all of mine manually with the
>>> help of auto-completion from Emacs, which is the most important feature IMO)
>>>
>>
>> Huh. Yes, I'll definitely have to look into the emacs mode. I assumed
>> once it was in ledger format it would be *a lot* harder to navigate around
>> vs. just doing it while it's already in a spreadsheet format.
>>
>
> Definitely not, text is there for your pleasure. You typically organize
> your Ledger input file in the order that makes the most sense for you
> (minus some constraints: Ledger will report the transactions in the order
> they appear in the file and the balance assertions are computed as such.
> Beancount sorts everything by date so order doesn't matter).
>
>
>
> - Moving the resulting transactions to the right place in your file.
>>>
>>
>> I'll have to look into this more. I get that this is the ledger list...
>> but is beancount different in this respect? From reading your docs, it
>> sounded like beancount didn't care about order. Or are there 

Re: Getting started; assigning accounts to bank .csv data

[John Hendy]

On Monday, February 1, 2016 at 10:41:26 PM UTC-6, Martin Blais wrote:
>
> On Mon, Feb 1, 2016 at 1:13 PM, John Hendy  > wrote:
>
>> Greetings,
>>
>>
>> It's a fresh year and I've been seeing ledger come up on the Org-mode
>> mailing list for some time and decided to give it a try. I'm coming
>> from Moneydance and just wanted to get away from the tedious GUI
>> method of adding information, as well as have flexibility to generate
>> my own reports/visualizations with python or R, etc. [1]
>>
>> Consider that I'm about a week into reading through docs here and
>> there during evenings. My first step was going to be importing a
>> downloaded .csv from my bank to get started. I'm still trying to
>> verify I get the terminology, so I'll use this from the manual:
>>
>> From 5.1 Basic format:
>> ```
>> This transaction has a date, a payee or description, a target account
>> (the first posting), and a source account (the second posting). Each
>> posting specifies what action is taken related to that account.
>> ```
>>
>> From 7.2.1.2 The convert command:
>> ```
>> The fields ledger can recognize contain these case-insensitive strings
>> date, posted, code, payee or desc or description, amount, cost,total,
>> and note.
>> ```
>>
>> For my purposes, I import my finances primarily to "categorize" (what
>> I believe here is called adding an account) and assign a payee so that
>> I can track my spending against a budget. So, I'm surprised there's no
>> special column keyword I can add for "account". It appears that all I
>> can do is pass, say, `--account "assets:checking"` to have ledger know
>> it's against assets:checking? Is that correct?
>>
>> From trying to google "import csv account ledger" or similar
>> variations, I've been surprised that the only tools to do something
>> like this appear to be interactive one-trans-at-a-time programs like
>> icsv2ledger and reckon (granted, they can learn or follow rules). I
>> could quickly go through my bank's .csv and add exp:food:dining,
>> exp:auto:fuel to my ~100 transactions a month and have those imported
>> just like the other column data.
>>
>
>
Thanks for the awesome reply!
 

> Keep in mind that part of the process of importing (they like call it 
> "reconciling") involves
> - Manually reviewing the transactions for correctness or fraud
>

I'll get there. For better or worse, I take the downloaded bank .csv as 
"truth" and am mostly interested in getting a better handle on what my 
money is used on, budgeting, planning, etc.
 

> - Merging new transactions with previous transactions imported from the 
> other side (e.g. a payment from a bank account to pay off on'es credit card 
> will typically be imported from both the bank AND credit card accounts; you 
> must merge the corresponding transactions together)
>

Definitely. Moneydance allowed me to input an account, which would "link" 
the transaction. Then I'd have to delete or merge the other account's 
record of the same transaction.
 

> - Assigning the right category (you can automate this with a script I 
> suppose; frankly it's not much work, I do all of mine manually with the 
> help of auto-completion from Emacs, which is the most important feature IMO)
>

Huh. Yes, I'll definitely have to look into the emacs mode. I assumed once 
it was in ledger format it would be *a lot* harder to navigate around vs. 
just doing it while it's already in a spreadsheet format.
 

> - Moving the resulting transactions to the right place in your file.
>

I'll have to look into this more. I get that this is the ledger list... but 
is beancount different in this respect? From reading your docs, it sounded 
like beancount didn't care about order. Or are there other reasons (besides 
date) that one would have to move transactions around?
 

> - Verifying balances visually, or inserting a balance directive which 
> asserts what the final account balance should be (for correctness) after 
> the new transactions.
>
> If you do it often enough and you have editing chops, you get used to the 
> dance and it's a breeze.
> I think the fourth step can be hypothetically solved using heuristics.
>
>
>
> I feel like I must be missing something with respect to getting the
>> from/to accounts added to the bank data.
>>
>> Perhaps to take a step back...
>> - are the majority of folks writing their transactions by hand in ledger 
>> format?
>>
>
> Can't say about others, but for me I want to say that about half the 
> importing is semi-automatic.
> - Credit cards and banks import from downloads but I need to categorize 
> manually (as described above), fairly good quality downloads.
> - Investment accounts fully automated buys but I need to manually edit 
> sales in some accounts. Great quality of downloads.
> - Payroll stubs and vesting and a few other things are provided only as 
> PDFs and I don't bother trying to extract (though I've made some headway 
> towards this, it's incomplete; it turns out fully 

Re: Getting started; assigning accounts to bank .csv data

[Martin Blais]

On Tue, Feb 2, 2016 at 10:48 PM, John Hendy  wrote:
>
> On Monday, February 1, 2016 at 10:41:26 PM UTC-6, Martin Blais wrote:
>
>> On Mon, Feb 1, 2016 at 1:13 PM, John Hendy  wrote:
>>
>>> Greetings,
>>>
>>>
>>> It's a fresh year and I've been seeing ledger come up on the Org-mode
>>> mailing list for some time and decided to give it a try. I'm coming
>>> from Moneydance and just wanted to get away from the tedious GUI
>>> method of adding information, as well as have flexibility to generate
>>> my own reports/visualizations with python or R, etc. [1]
>>>
>>> Consider that I'm about a week into reading through docs here and
>>> there during evenings. My first step was going to be importing a
>>> downloaded .csv from my bank to get started. I'm still trying to
>>> verify I get the terminology, so I'll use this from the manual:
>>>
>>> From 5.1 Basic format:
>>> ```
>>> This transaction has a date, a payee or description, a target account
>>> (the first posting), and a source account (the second posting). Each
>>> posting specifies what action is taken related to that account.
>>> ```
>>>
>>> From 7.2.1.2 The convert command:
>>> ```
>>> The fields ledger can recognize contain these case-insensitive strings
>>> date, posted, code, payee or desc or description, amount, cost,total,
>>> and note.
>>> ```
>>>
>>> For my purposes, I import my finances primarily to "categorize" (what
>>> I believe here is called adding an account) and assign a payee so that
>>> I can track my spending against a budget. So, I'm surprised there's no
>>> special column keyword I can add for "account". It appears that all I
>>> can do is pass, say, `--account "assets:checking"` to have ledger know
>>> it's against assets:checking? Is that correct?
>>>
>>> From trying to google "import csv account ledger" or similar
>>> variations, I've been surprised that the only tools to do something
>>> like this appear to be interactive one-trans-at-a-time programs like
>>> icsv2ledger and reckon (granted, they can learn or follow rules). I
>>> could quickly go through my bank's .csv and add exp:food:dining,
>>> exp:auto:fuel to my ~100 transactions a month and have those imported
>>> just like the other column data.
>>>
>>
>>
> Thanks for the awesome reply!
>
>
>> Keep in mind that part of the process of importing (they like call it
>> "reconciling") involves
>> - Manually reviewing the transactions for correctness or fraud
>>
>
> I'll get there. For better or worse, I take the downloaded bank .csv as
> "truth" and am mostly interested in getting a better handle on what my
> money is used on, budgeting, planning, etc.
>
>
>> - Merging new transactions with previous transactions imported from the
>> other side (e.g. a payment from a bank account to pay off on'es credit card
>> will typically be imported from both the bank AND credit card accounts; you
>> must merge the corresponding transactions together)
>>
>
> Definitely. Moneydance allowed me to input an account, which would "link"
> the transaction. Then I'd have to delete or merge the other account's
> record of the same transaction.
>

BTW, there are some ideas around about automatically merging two incomplete
transactions. This problem is the dual of solving the issue of settlement
dates, i.e., the problem being that the dates of each of the two sides may
settlement on different days.
See http://furius.ca/beancount/doc/proposal-settlement for some ruminations
and scour the mailing-list, there is more discussion about this.



- Assigning the right category (you can automate this with a script I
>> suppose; frankly it's not much work, I do all of mine manually with the
>> help of auto-completion from Emacs, which is the most important feature IMO)
>>
>
> Huh. Yes, I'll definitely have to look into the emacs mode. I assumed once
> it was in ledger format it would be *a lot* harder to navigate around vs.
> just doing it while it's already in a spreadsheet format.
>

Definitely not, text is there for your pleasure. You typically organize
your Ledger input file in the order that makes the most sense for you
(minus some constraints: Ledger will report the transactions in the order
they appear in the file and the balance assertions are computed as such.
Beancount sorts everything by date so order doesn't matter).



- Moving the resulting transactions to the right place in your file.
>>
>
> I'll have to look into this more. I get that this is the ledger list...
> but is beancount different in this respect? From reading your docs, it
> sounded like beancount didn't care about order. Or are there other reasons
> (besides date) that one would have to move transactions around?
>

In Ledger, the reporting is done in file order. Balance assertions as well.
In Beancount, order is by date, so you don't have to care about how you
organize them.
I think - but I'm not 100% sure - that most Ledger users must store their
input file by section, and in each section in date 

Getting started; assigning accounts to bank .csv data

[John Hendy]

Greetings,


It's a fresh year and I've been seeing ledger come up on the Org-mode
mailing list for some time and decided to give it a try. I'm coming
from Moneydance and just wanted to get away from the tedious GUI
method of adding information, as well as have flexibility to generate
my own reports/visualizations with python or R, etc. [1]

Consider that I'm about a week into reading through docs here and
there during evenings. My first step was going to be importing a
downloaded .csv from my bank to get started. I'm still trying to
verify I get the terminology, so I'll use this from the manual:

>From 5.1 Basic format:
```
This transaction has a date, a payee or description, a target account
(the first posting), and a source account (the second posting). Each
posting specifies what action is taken related to that account.
```

>From 7.2.1.2 The convert command:
```
The fields ledger can recognize contain these case-insensitive strings
date, posted, code, payee or desc or description, amount, cost,total,
and note.
```

For my purposes, I import my finances primarily to "categorize" (what
I believe here is called adding an account) and assign a payee so that
I can track my spending against a budget. So, I'm surprised there's no
special column keyword I can add for "account". It appears that all I
can do is pass, say, `--account "assets:checking"` to have ledger know
it's against assets:checking? Is that correct?

>From trying to google "import csv account ledger" or similar
variations, I've been surprised that the only tools to do something
like this appear to be interactive one-trans-at-a-time programs like
icsv2ledger and reckon (granted, they can learn or follow rules). I
could quickly go through my bank's .csv and add exp:food:dining,
exp:auto:fuel to my ~100 transactions a month and have those imported
just like the other column data.

I feel like I must be missing something with respect to getting the
from/to accounts added to the bank data.

Perhaps to take a step back...
- are the majority of folks writing their transactions by hand in ledger format?
- is there some better way to import bulk data (e.g. via ledger's
convert function) and post-edit once it's in ledger format? It seemed
a .csv in LO calc was pretty convenient vs. scrolling through a long
text file
- any other pointers along the above lines would be most welcome.


I tried to search the list for more of this sort of question, so
forgive me if I've missed something. Replying with links pointing me
in the right direction would be plenty sufficient if this has already
been discussed!


Thanks!
John


[1] http://moneydance.com/

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"Ledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ledger-cli+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Getting started; assigning accounts to bank .csv data

[Craig Earls]

I have a custom elisp mode that sucks in csv from my bank and sticks it in
my ledger. The emacs ledger mode has enough built in autocompletion that
hand rntering a few xavts here and there isnt onerous. Its all about the
editor.

On Monday, February 1, 2016, John Hendy  wrote:

> Greetings,
>
>
> It's a fresh year and I've been seeing ledger come up on the Org-mode
> mailing list for some time and decided to give it a try. I'm coming
> from Moneydance and just wanted to get away from the tedious GUI
> method of adding information, as well as have flexibility to generate
> my own reports/visualizations with python or R, etc. [1]
>
> Consider that I'm about a week into reading through docs here and
> there during evenings. My first step was going to be importing a
> downloaded .csv from my bank to get started. I'm still trying to
> verify I get the terminology, so I'll use this from the manual:
>
> From 5.1 Basic format:
> ```
> This transaction has a date, a payee or description, a target account
> (the first posting), and a source account (the second posting). Each
> posting specifies what action is taken related to that account.
> ```
>
> From 7.2.1.2 The convert command:
> ```
> The fields ledger can recognize contain these case-insensitive strings
> date, posted, code, payee or desc or description, amount, cost,total,
> and note.
> ```
>
> For my purposes, I import my finances primarily to "categorize" (what
> I believe here is called adding an account) and assign a payee so that
> I can track my spending against a budget. So, I'm surprised there's no
> special column keyword I can add for "account". It appears that all I
> can do is pass, say, `--account "assets:checking"` to have ledger know
> it's against assets:checking? Is that correct?
>
> From trying to google "import csv account ledger" or similar
> variations, I've been surprised that the only tools to do something
> like this appear to be interactive one-trans-at-a-time programs like
> icsv2ledger and reckon (granted, they can learn or follow rules). I
> could quickly go through my bank's .csv and add exp:food:dining,
> exp:auto:fuel to my ~100 transactions a month and have those imported
> just like the other column data.
>
> I feel like I must be missing something with respect to getting the
> from/to accounts added to the bank data.
>
> Perhaps to take a step back...
> - are the majority of folks writing their transactions by hand in ledger
> format?
> - is there some better way to import bulk data (e.g. via ledger's
> convert function) and post-edit once it's in ledger format? It seemed
> a .csv in LO calc was pretty convenient vs. scrolling through a long
> text file
> - any other pointers along the above lines would be most welcome.
>
>
> I tried to search the list for more of this sort of question, so
> forgive me if I've missed something. Replying with links pointing me
> in the right direction would be plenty sufficient if this has already
> been discussed!
>
>
> Thanks!
> John
>
>
> [1] http://moneydance.com/
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "Ledger" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ledger-cli+unsubscr...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Craig, Corona De Tucson, AZ
enderw88.wordpress.com

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"Ledger" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ledger-cli+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Getting started; assigning accounts to bank .csv data

[Martin Blais]

On Mon, Feb 1, 2016 at 1:13 PM, John Hendy  wrote:

> Greetings,
>
>
> It's a fresh year and I've been seeing ledger come up on the Org-mode
> mailing list for some time and decided to give it a try. I'm coming
> from Moneydance and just wanted to get away from the tedious GUI
> method of adding information, as well as have flexibility to generate
> my own reports/visualizations with python or R, etc. [1]
>
> Consider that I'm about a week into reading through docs here and
> there during evenings. My first step was going to be importing a
> downloaded .csv from my bank to get started. I'm still trying to
> verify I get the terminology, so I'll use this from the manual:
>
> From 5.1 Basic format:
> ```
> This transaction has a date, a payee or description, a target account
> (the first posting), and a source account (the second posting). Each
> posting specifies what action is taken related to that account.
> ```
>
> From 7.2.1.2 The convert command:
> ```
> The fields ledger can recognize contain these case-insensitive strings
> date, posted, code, payee or desc or description, amount, cost,total,
> and note.
> ```
>
> For my purposes, I import my finances primarily to "categorize" (what
> I believe here is called adding an account) and assign a payee so that
> I can track my spending against a budget. So, I'm surprised there's no
> special column keyword I can add for "account". It appears that all I
> can do is pass, say, `--account "assets:checking"` to have ledger know
> it's against assets:checking? Is that correct?
>
> From trying to google "import csv account ledger" or similar
> variations, I've been surprised that the only tools to do something
> like this appear to be interactive one-trans-at-a-time programs like
> icsv2ledger and reckon (granted, they can learn or follow rules). I
> could quickly go through my bank's .csv and add exp:food:dining,
> exp:auto:fuel to my ~100 transactions a month and have those imported
> just like the other column data.
>

Keep in mind that part of the process of importing (they like call it
"reconciling") involves
- Manually reviewing the transactions for correctness or fraud
- Merging new transactions with previous transactions imported from the
other side (e.g. a payment from a bank account to pay off on'es credit card
will typically be imported from both the bank AND credit card accounts; you
must merge the corresponding transactions together)
- Assigning the right category (you can automate this with a script I
suppose; frankly it's not much work, I do all of mine manually with the
help of auto-completion from Emacs, which is the most important feature IMO)
- Moving the resulting transactions to the right place in your file.
- Verifying balances visually, or inserting a balance directive which
asserts what the final account balance should be (for correctness) after
the new transactions.

If you do it often enough and you have editing chops, you get used to the
dance and it's a breeze.
I think the fourth step can be hypothetically solved using heuristics.



I feel like I must be missing something with respect to getting the
> from/to accounts added to the bank data.
>
> Perhaps to take a step back...
> - are the majority of folks writing their transactions by hand in ledger
> format?
>

Can't say about others, but for me I want to say that about half the
importing is semi-automatic.
- Credit cards and banks import from downloads but I need to categorize
manually (as described above), fairly good quality downloads.
- Investment accounts fully automated buys but I need to manually edit
sales in some accounts. Great quality of downloads.
- Payroll stubs and vesting and a few other things are provided only as
PDFs and I don't bother trying to extract (though I've made some headway
towards this, it's incomplete; it turns out fully automating table
extraction from PDF isn't trivial. The best OSS solution is TabulaPDF by
far but you still need to manually identify where the table is).
- Cash transactions: I have to enter those by hand. I only book non-food
expenses as individual transactions directly, and for food maybe once every
six months I'll count my wallet balance and insert one transaction per
month to debit away the cash account toward food. If you do this, you end
up with surprisingly little transactions to book manually, maybe a
few/week. I suppose it could depend on lifestyle choices.

It takes me less than 1 hour/week to run through the active accounts,
usually first thing Saturday morning when I get up. Most of the pain is
logging with user/passwords into the various institutions and clicking the
right buttons to generate the downloaded files. Extraction and filing is
automated using importers I wrote against LedgerHub. Less active accounts
are updated every quarter or when I feel like it.




> - is there some better way to import bulk data (e.g. via ledger's
> convert function) and post-edit once it's in ledger format? It seemed