Thank you for sharing, Martin. This is a reasonably well-written and fair enough review, though with a bit of a snarky tone and perhaps some misconceptions (though I can't blame the author for not seeing through all the nooks and crannies in just a few days exploration). Here's a response that attempts to address the points made in the article.
First, I'm impressed that he managed to import his QuickBooks data in just a few hours. I didn't realize it had become that easy. RECONCILIATION. The reviewer mentioned the lack of reconciliation as a "nearly fatal flaw." What he's referring to is not specifically the absence of a flagging mechanism (because we have one) but rather the lack of support for different dates in a single transaction. A proposal for solving this was sketched here: https://docs.google.com/document/d/1x0qqWGRHi02ef-FtUW172SHkdJ8quOZD-Xli7r4Nl_k/ While in practice fudging one of the dates has worked fine for personal finances (and I've always advocated for doing that until that's resolved), I can see that for more serious bookkeeping needs - where one may desire to produce a precise journal of postings for an account, with all the dates matching exactly with those of the institution - it would be important to support it (and I'd also like to start using it like that too for myself). It is also a recurring topic on the list from people who've used commercial accounting software. The proposal mentioned above would work fine with transfer accounts (I actually don't know how "real" accounting software deals with this behind the scenes, transfer accounts too?) and has little in the way of blockers -- it wouldn't add complexity to the rest of the system (though it would split up a lot of transactions into two transactions). Basically adding support for inputting per-posting alternate dates and having a plugin split up those transactions to post to/from one (or more) transfer account(s). Also supporting the /merging/ of transactions with a single posting each would be useful too (just another way to achieve the same thing). The reviewer does mention "the reconciliation pass." This obviously comes from a history of working without assertions. To a large extent, assertions obviate the need for this, but he missed that you can use flags for this anyway (that's their main purpose, e.g. using '*' and '!' to make transactions are correct or incomplete) and suggests the use of metadata instead, which is a bit overkill in terms of syntax. To go through the set of unreconciliated transactions that's nothing to build, one can just use a SQL command, something like SELECT location WHERE date >= 2019-02-01 and flag ='!' and iterate using "next-error" in Emacs. Perhaps that should be documented better. We could also build some features to ensure that all transactions before some specific date are flagged done and correct (trivially easy). CHECKS & TRANSACTION NUMBERS. For those you can definitely use links or tags, or metadata and the reviewer mentions this possibility, but alludes that it would be work to "actually write the accounting system." I'd like to know what kind of processes he'd like to see supported. I'm not too familiar with what type of support is provided for checks, for example, all I can imagine is (a) checking that no two check numbers are repeated, and/or that all consecutive numbers from the smallest one are present, or (b) special reporting by order of check number. Both of these ideas would be very simple to implement as plugins, but I'm not sure whether they are canonical enough to provide as default? (I'd imagine if that were the case over the last 10 years someone would have made noises about it.) In any case, those would be easy to support. Generally speaking we can't possibly cover all the cases - but if someone with more knowledge of e.g. QuickBooks would like to prepare a list of essential functionality, we can consider building plugins for those features in the repo. IGNORED LINES. I think that comment stems from outdated documentation. Org-mode / Markdown start-line characters are now ignored explicitly, see here: https://bitbucket.org/blais/beancount/src/fa1edde3bcd02a277fac193f460a39c9a1461161/beancount/parser/lexer.l#lines-173 So you can probably sleep at night. But if you're concerned with this I'd like to challenge you to come up with examples of silently ignored input and file a ticket, I would certainly fix this ASAP. ACCOUNTING vs. BOOKKEEPING. The reviewer mentions that many features one would expect from an accounting system are absent. That is correct -- the word "accounting" is used rather loosely here (and in all PTA systems) and it would be more fair to call it a "book-keeping" system. It's a parser and a good, simple schema that applies a simple constraint and generates a stream of transactions, + a mish-mash of various loose functionalities built on top. Generating invoices would require someone to build an item (inventory?) tracking subsystem and client information database. These are considered out of scope for Beancount, but I can easily imagine something that would integrate well with it. Besides, we have users all over the world, and building such a system, with the inevitable myriad features and variants it would involve, would be worthy of a full project on its own. Good design (e.g. UNIX) is that we build simple tools that do one thing and just one thing well yet allow composition with other tools and this is precisely the point here. Someone ought to build a free, OSS invoicing system and the integration with our book-keeping system would be easy. A similar comment applies for the reviewer's notice of the absence of customers or vendors. I'd have expected an LWN reviewer to see through that. I think the reviewer's comments about "tracking invoices" and the absence of a dedicated concept for accounts payable / receivable refers to ensuring that invoices have been paid. Since invoices aren't present as concepts in the ledger, AP / AR aren't treated in any other way than other accounts. However, I've shown in the past that you can build your own custom tracking by providing a 90-line plugin that does that: https://bitbucket.org/blais/beancount/src/default/beancount/plugins/tag_pending.py Again, given it's so simple to add that in via a small piece of code I have to wonder whether it's worthwhile to add a first-class concept for that in Beancount itself. Maybe writing a doc about one would go about doing that is the best way to proceed? Maybe invoicing / closing an invoice ought to be first-class concepts in the schema? The author mentions the difficulty of handling duplicate transactions; that's probably from a thread on the mailing-list where we discussed *automatically detecting* such duplicates, which is more demanding than explicitly handling that via unique numbers. By implementing very simple constraints on links on transactions - treating a subset of link names as "unique transaction numbers" if you will - one would be able to achieve this with a 20 LOC plugin. Again, I can imagine many variants for this, so it's best left to a bit of custom code IMO for your own interpretation of metadata. Maybe we should have a built-in plugin provided that does that (e.g. recognize some metadata or "regexp on link" pattern and ensure each value occurs no more than once on the stream - easy). Maybe what we need is a new, middle layer, that builds in a bunch of these concepts at once on top of the existing schema. Maybe plugins aren't formal enough. REPORTING. The author mentions that it is impossible to restrict a report to a given period of time; that is incorrect. He must have missed that part from my long intro: https://docs.google.com/document/d/100tGcA4blh6KSXPRGCZpUlyxaRUwFHEvnz_k9DyZFn4/ where I describe how I translate the process of truncation of the past and future as "open" and "close" operations on the stream of directives, and the associated features on the SQL syntax where you can trigger these from a query. One of the design aspects of Beancount is that we can always defer period restrictions to reports and keep reading the full stream of transactions over all time every time. What may have happened here is that the now more-or-less obsolete bean-web tool and bean-report tools do lack that capability and the reviewer must have stumbled upon that. In the next version, these will be removed and the SQL capability will be expanded (and forked to a separate project). Being able to report over any period of time is the norm. About the slowness: this is true, it's a bit slow (the cache helps, but it's still slightly frustrating), and this will be handled in the next major version. (I think of the next version as essentially a rewrite in 3 parts: (1) rewrite just the core in C++ keeping the very same schema and same Python API but with protobufs instead of namedtuples as storage (cross language), (2) fork the SQL query tool to its own project, adding missing features and making it able to work on any data, not just Beancount, and (3) delete much of the custom reporting code, deferring users to either Fava or using the SQL tool (with extra whizbang) instead. Moreover, I'd like to move that next version to a development model more open to contributions, where I reduce the ruthlessly-unchanging and fiercely-defended-against-changes part to a small core written in C++ and the rest opens up more liberally than I have in the past, maybe even move to github, etc.) The author mentions that no free accounting systems generate government-mandated forms; you will never see any free software doing that. There are simply too many variants of forms, too much data that isn't part of the transaction flow, too many countries, etc. and most importantly, it's just not a sexy job. That's why companies like Intuit have thousands of developers being paid to process the minute details some obscure tax rule in a particular country for a particular year and figure out how to map some specific constrainted subset of accounts in their ledger to produce some number in some form, etc. Not Beancount nor any of the other PTA systems will ever do that. Nobody in their right mind would work on that on their free time on the weekend. I'd never expect that from any free system in the first place and I have to wonder why the reviewer did. What I think we might see instead, is a generic PDF overlay project, where one could combine a PDF file - from a government form - with an associated template with some input file to insert text where the template specifies it should (i..e template says "field 'name' is at <coordinates>", input file says "name = 'Joe'"). This could be easily integrated with something like Beancount by writing a script to pull the right numbers to generate the input file and then produce the PDF. That's a conceivable way such form generation could proceed in the OSS world (I started something like that a little while ago, but time is scarce). I enjoyed the review, thanks for sharing it. Cheers, On Fri, Apr 12, 2019 at 1:42 AM Martin Michlmayr <[email protected]> wrote: > LWN has reviewed beancount to see if it would fit their needs: > > https://lwn.net/SubscriberLink/785553/39167e23980e7e4c/ > > -- > Martin Michlmayr > https://www.cyrius.com/ > > -- > You received this message because you are subscribed to the Google Groups > "Beancount" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/beancount/20190412054207.GF1719%40jirafa.cyrius.com > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAK21%2BhMLj2BX0%3DdVKvi5zFczKFB%3DX1e-dub-shSGkT8VHjyd3Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
