Re: Branching transaction import logic (i.e. partially automating imports)

[Red S]

Red - do you know if there's a way to have smart-importer mark (by tag or 
metadata) the transactions that it updates? If not I can probably just mark 
the ones categorized via rules, i.e. those not processed by smart import!


I don't know unfortunately, but doing what you said (adding metadata to the 
ones you categorized) would  be trivial, and that's the way I'd go.

-- 
You received this message because you are subscribed to the Google Groups 
"Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beancount+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beancount/cdc7f1b3-5cd7-4eb1-9480-633e37100d8dn%40googlegroups.com.

Re: Branching transaction import logic (i.e. partially automating imports)

[Daniel Farnand]

Thank you all for super helpful answers!

I hadn't realized that smart importer wouldn't touch already categorized
transactions (obvious in retrospect though!). Thinking about it with that
in mind, I'm going to give it a try: import -> rules -> smart_importer.
>From there I'll pass everything to some other code to show aggregate info
and highlight anomalies (probably ad hoc after the import for now, though
I'll look into maybe adding it as a second hook with code like what Martin
showed).

Red - do you know if there's a way to have smart-importer mark (by tag or
metadata) the transactions that it updates? If not I can probably just mark
the ones categorized via rules, i.e. those not processed by smart import!

On Sat, Feb 10, 2024 at 10:03 AM Martin Blais  wrote:

> Here's how I do it (from my import script):
>
>
>
> def process_entry(entry):
> ... do something...
> return entry
>
>
> def process_extracted_entries(extracted_entries_list, ledger_entries):
> """Filter the extracted entries to save on time."""
> return [(filename, [process_entry(entry) for entry in entries],
> account, importer)
> for filename, entries, account, importer in
> extracted_entries_list]
>
>
> if __name__ == '__main__':
> main = Ingest(importers, hooks = [process_extracted_entries])
> main()
>
>
>
> On Fri, Feb 9, 2024 at 3:46 PM Danny  wrote:
>
>> *Background*
>>
>> I've been using beancount for a few years now. I just have a couple
>> credits cards and a bank account, nothing especially complex, but I feel
>> secure knowing I have a registry of where all my money has gone. Also the
>> process of getting transactions into beancount is my check on spending,
>> letting me notice anything suspicious.
>>
>> However, its just way too labor intensive. I already use
>> beancount-import, but still get bogged down in hundreds of $2.90 subway
>> payments, the grocery store, and sandwiches from the same handful of places.
>>
>> *What I'm Looking For*
>>
>> I need a less time-consuming workflow. I discovered Red's five minute
>> ledger, and agree completely with the philosophy. However I think I need a
>> way to separate transactions from any given account into two separate
>> streams.
>>
>> To better illustrate, this is my ideal pipeline:
>>
>>1. Download transactions manually or automatically where possible
>>(csv and ofx)
>>2. Run code that has a set of predefined expense category rules (e.g.
>>amazon automatically to a zero-sum category, grocery store below certain
>>dollar value)
>>3. Separate the categorized transactions and pass the remaining ones
>>to beancount-import
>>4. Write everything to the ledger like normal
>>
>> I haven't found any examples of branching the transaction pipeline like
>> this, so my question is whether its even plausible within the framework of
>> beancount importers. My back up plan is to write a more or less hardcoded
>> script that will do it all, but I'm hoping for a more flexible approach!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Beancount" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to beancount+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/beancount/1d463106-57c7-447c-b374-087ab60943ddn%40googlegroups.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Beancount" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/beancount/OxWzJz46yvo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> beancount+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/beancount/CAK21%2BhOgxJu8-83Nb%3DfNSQp-7VfC0gsdX8d-%2BxL%3D6jJBOz%2B_1Q%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beancount+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beancount/CAOeb5QocUaAhvYaPyBTWNTfwpF%3DMjbO95J0s_ojCKqGndPjO-Q%40mail.gmail.com.

Re: Branching transaction import logic (i.e. partially automating imports)

[Martin Blais]

Here's how I do it (from my import script):



def process_entry(entry):
... do something...
return entry


def process_extracted_entries(extracted_entries_list, ledger_entries):
"""Filter the extracted entries to save on time."""
return [(filename, [process_entry(entry) for entry in entries],
account, importer)
for filename, entries, account, importer in
extracted_entries_list]


if __name__ == '__main__':
main = Ingest(importers, hooks = [process_extracted_entries])
main()



On Fri, Feb 9, 2024 at 3:46 PM Danny  wrote:

> *Background*
>
> I've been using beancount for a few years now. I just have a couple
> credits cards and a bank account, nothing especially complex, but I feel
> secure knowing I have a registry of where all my money has gone. Also the
> process of getting transactions into beancount is my check on spending,
> letting me notice anything suspicious.
>
> However, its just way too labor intensive. I already use beancount-import,
> but still get bogged down in hundreds of $2.90 subway payments, the grocery
> store, and sandwiches from the same handful of places.
>
> *What I'm Looking For*
>
> I need a less time-consuming workflow. I discovered Red's five minute
> ledger, and agree completely with the philosophy. However I think I need a
> way to separate transactions from any given account into two separate
> streams.
>
> To better illustrate, this is my ideal pipeline:
>
>1. Download transactions manually or automatically where possible (csv
>and ofx)
>2. Run code that has a set of predefined expense category rules (e.g.
>amazon automatically to a zero-sum category, grocery store below certain
>dollar value)
>3. Separate the categorized transactions and pass the remaining ones
>to beancount-import
>4. Write everything to the ledger like normal
>
> I haven't found any examples of branching the transaction pipeline like
> this, so my question is whether its even plausible within the framework of
> beancount importers. My back up plan is to write a more or less hardcoded
> script that will do it all, but I'm hoping for a more flexible approach!
>
> --
> You received this message because you are subscribed to the Google Groups
> "Beancount" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to beancount+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/beancount/1d463106-57c7-447c-b374-087ab60943ddn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beancount+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beancount/CAK21%2BhOgxJu8-83Nb%3DfNSQp-7VfC0gsdX8d-%2BxL%3D6jJBOz%2B_1Q%40mail.gmail.com.

Re: Copy-pasting multiline (multiple lines) query into bean-query

[Martin Blais]

This is a bash question. Look at how multiple lines are handled in bash
(should be lots of info online).

On Sat, Feb 10, 2024 at 9:24 AM Dan Andersson 
wrote:

> Hi,
>
> After doing bean-query foo.beancount to enter the interactive session, can
> I somehow copy-paste multiple lines in there?
>
> It seems like multi-line queries (e.g. the examples in the documentation),
> when pasted in there, always get split into one command per line, resulting
> in errors.
>
> Do I always have to combine those lines first, before pasting them?
>
> Thanks,
> DA
>
> --
> You received this message because you are subscribed to the Google Groups
> "Beancount" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to beancount+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/beancount/50b1d037-b464-4838-ae46-58ffc250e8b8n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beancount+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beancount/CAK21%2BhPv6nMsT1ES8Td4U1BJv2hqi%3DQd3bMaKO-e2Hj605kscQ%40mail.gmail.com.

Copy-pasting multiline (multiple lines) query into bean-query

[Dan Andersson]

Hi,

After doing bean-query foo.beancount to enter the interactive session, can 
I somehow copy-paste multiple lines in there?

It seems like multi-line queries (e.g. the examples in the documentation), 
when pasted in there, always get split into one command per line, resulting 
in errors.

Do I always have to combine those lines first, before pasting them?

Thanks,
DA

-- 
You received this message because you are subscribed to the Google Groups 
"Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beancount+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beancount/50b1d037-b464-4838-ae46-58ffc250e8b8n%40googlegroups.com.

Re: Branching transaction import logic (i.e. partially automating imports)

[Red S]

Thinking about your question further: I find that when I import 
transactions, credit cards are the only ones that deal with expenses and 
need categorization. I find that smart_importer gets almost all of them 
right. I do skim through the import as a part of my workflow, but it takes 
less than a minute to rapidly look through a month's data (say, a 100 
transactions), and find the occasional one that needs to be fixed.

Yet another way to address this problem is to glance at my monthly 
expenditure categories in a report (I use Fava), where big 
miscategorizations (a rare occurrence) stand out fairly obviously.

On Saturday, February 10, 2024 at 1:35:09 AM UTC-8 Red S wrote:

> I'm not familiar with beancount-importer, but this should work out of the 
> box if you use smart_importer, with nothing for you to really do. As it 
> says :
>
> smart_importer only works by appending onto incomplete single-legged 
> postings
>
> So in the importer you write, simply leave the ones you want uncategorized 
> with no further action, and smart_importer will categorize them, and leave 
> the other ones untouched.
>
> Trivial to do with beancount_reds_importers too: simply override this 
> method 
> 
>  
> to return either your pre-defined postings, or None if you want it to be 
> auto-categorized by smart_importer.
> ​
> On Friday, February 9, 2024 at 12:46:03 PM UTC-8 Danny wrote:
>
>> *Background*
>>
>> I've been using beancount for a few years now. I just have a couple 
>> credits cards and a bank account, nothing especially complex, but I feel 
>> secure knowing I have a registry of where all my money has gone. Also the 
>> process of getting transactions into beancount is my check on spending, 
>> letting me notice anything suspicious.
>>
>> However, its just way too labor intensive. I already use 
>> beancount-import, but still get bogged down in hundreds of $2.90 subway 
>> payments, the grocery store, and sandwiches from the same handful of places.
>>
>> *What I'm Looking For*
>>
>> I need a less time-consuming workflow. I discovered Red's five minute 
>> ledger, and agree completely with the philosophy. However I think I need a 
>> way to separate transactions from any given account into two separate 
>> streams.
>>
>> To better illustrate, this is my ideal pipeline:
>>
>>1. Download transactions manually or automatically where possible 
>>(csv and ofx)
>>2. Run code that has a set of predefined expense category rules (e.g. 
>>amazon automatically to a zero-sum category, grocery store below certain 
>>dollar value)
>>3. Separate the categorized transactions and pass the remaining ones 
>>to beancount-import
>>4. Write everything to the ledger like normal
>>
>> I haven't found any examples of branching the transaction pipeline like 
>> this, so my question is whether its even plausible within the framework of 
>> beancount importers. My back up plan is to write a more or less hardcoded 
>> script that will do it all, but I'm hoping for a more flexible approach!
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beancount+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beancount/dbc6c223-e00f-4042-a2d9-0b07da7311a3n%40googlegroups.com.

Re: Branching transaction import logic (i.e. partially automating imports)

[Red S]

I'm not familiar with beancount-importer, but this should work out of the 
box if you use smart_importer, with nothing for you to really do. As it says 
:

smart_importer only works by appending onto incomplete single-legged 
postings

So in the importer you write, simply leave the ones you want uncategorized 
with no further action, and smart_importer will categorize them, and leave 
the other ones untouched.

Trivial to do with beancount_reds_importers too: simply override this method 

 
to return either your pre-defined postings, or None if you want it to be 
auto-categorized by smart_importer.
​
On Friday, February 9, 2024 at 12:46:03 PM UTC-8 Danny wrote:

> *Background*
>
> I've been using beancount for a few years now. I just have a couple 
> credits cards and a bank account, nothing especially complex, but I feel 
> secure knowing I have a registry of where all my money has gone. Also the 
> process of getting transactions into beancount is my check on spending, 
> letting me notice anything suspicious.
>
> However, its just way too labor intensive. I already use beancount-import, 
> but still get bogged down in hundreds of $2.90 subway payments, the grocery 
> store, and sandwiches from the same handful of places.
>
> *What I'm Looking For*
>
> I need a less time-consuming workflow. I discovered Red's five minute 
> ledger, and agree completely with the philosophy. However I think I need a 
> way to separate transactions from any given account into two separate 
> streams.
>
> To better illustrate, this is my ideal pipeline:
>
>1. Download transactions manually or automatically where possible (csv 
>and ofx)
>2. Run code that has a set of predefined expense category rules (e.g. 
>amazon automatically to a zero-sum category, grocery store below certain 
>dollar value)
>3. Separate the categorized transactions and pass the remaining ones 
>to beancount-import
>4. Write everything to the ledger like normal
>
> I haven't found any examples of branching the transaction pipeline like 
> this, so my question is whether its even plausible within the framework of 
> beancount importers. My back up plan is to write a more or less hardcoded 
> script that will do it all, but I'm hoping for a more flexible approach!
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beancount+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beancount/7fa55de6-7881-40d3-a34f-73011d6c37cbn%40googlegroups.com.