A key question is whether it would be possible to unify the beancount-import concept of a source with the beancount concept of an ingester.
The main differences are: - beancount-import expects the source to generate full transactions, rather than partial transactions, but may specify some accounts as Expenses:FIXME to indicate an unknown account. Beancount ingesters generate partial transactions with missing postings if accounts are unknown. - beancount-import additionally expects data sources to: 1. automatically avoid importing already-imported transactions in a reliable, non-heuristic way 2. Optionally indicate which postings are considered cleared for a set of accounts for which it declares itself authoritative, 3. Optionally specify how to extract features from postings/transactions for automatic prediction of unknown accounts. All of these extra things are done by adding source-specific metadata to the generated transactions. It is easy to convert a partial transaction into a full transaction as expected by beancount-import by adding an extra Expenses:FIXME posting with the residual. The extra metadata required cannot be generated automatically, though. On Oct 4, 2018 18:27, "Martin Blais" <[email protected]> wrote: This looks great! :-) Thank you for sharing your workflow with everyone. (Link added to the contrib doc.) On Thu, Oct 4, 2018 at 7:53 PM Jeremy Maitin-Shepard <[email protected]> wrote: > I'd like to announce beancount-import, a tool for semi-automatically > importing transactions from external data sources, with support for merging > and reconciling imported transactions with each other and with existing > transactions in the beancount journal. The UI is web based. > > You can find detailed information, setup and configuration instructions, > and examples to run on test data at: > https://github.com/jbms/beancount-import > > This tool differs from the existing transaction import functionality in > beancount in several important ways: > - it includes metadata in imported entries that allow it to reliably > associate entries in the journal with external data, so that you don't need > to manually track what has already been imported. > - there is automatic prediction of unknown accounts (currently based on a > learned decision tree classifier) > - rather than operating in a purely append mode, there is a powerful > matching mechanism that can propose matches between existing transactions > and new imported transactions, which handles transfers between accounts, > manually entered transactions, and many other cases. Matches to existing > transactions are handled by editing the journal in place to add any > additional postings/metadata, which is accomplished safely using the > journal_editor module that is part of beancount-import, but which is also > useful independently. > > There is existing, well-tested support for a variety of data sources, > including OFX files (checking/savings/credit card/investment/retirement > accounts), downloaded Mint.com transactions, Venmo, Amazon.com order > invoices, and others. There is also a clean API for defining new data > sources. > > I'm also releasing the related package finance-dl, available at > https://github.com/jbms/finance-dl, which is useful for automatically > downloading data that can then be fed into beancount-import. This tool > currently supports a number of services including the OFX protocol, > Mint.com, Amazon.com, and Venmo, and can also be extended to support other > websites. > > This is a completely rewritten successor to the original, much more > limited beancount-import tool that I released several years ago. I've been > developing this tool over the past several years, and have been > successfully using it to maintain my beancount journal containing many > thousands of transactions. I finally got around to cleaning it up and > polishing it for release so that it may be useful to others. > > I know that there is some overlap with other transaction import tools > being developed. I'm very open to finding ways that we can > collaborate/combine our efforts. > > Regarding the name, if there is strong opposition to using beancount in > the name, I'm happy to change it to something else. > > -- > You received this message because you are subscribed to the Google Groups > "Beancount" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/beancount/CAKJfoCE_9_kEzfh75p9i76AzC9iThCEkeD%2BnWVR_qjDVtZ7fTQ%40mail.gmail.com > <https://groups.google.com/d/msgid/beancount/CAKJfoCE_9_kEzfh75p9i76AzC9iThCEkeD%2BnWVR_qjDVtZ7fTQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAK21%2BhMq5tj%2BKfXxuvTX%3DFnirEwQ-93w5ALXc1mm1Jvj_NBXTQ%40mail.gmail.com <https://groups.google.com/d/msgid/beancount/CAK21%2BhMq5tj%2BKfXxuvTX%3DFnirEwQ-93w5ALXc1mm1Jvj_NBXTQ%40mail.gmail.com?utm_medium=email&utm_source=footer> . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAKJfoCHBbFuFqohyaby96fhpS03disa7SJUnKYvCgkv5Gmc2-A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
