Hi Martin, The # char in cost basis is a new thing to me and I don't think the syntax is mentioned in the language syntax doc https://docs.google.com/document/d/1wAMVrKIA2qtRGmoVDSUBJGmYZSygUaR0uOMW1GV3YE0/edit# Could you document it there? You used `Assets:Account 100 MSFT {0 # 2000.00 USD}` in the example. I don't think the {{...}} syntax is mentioned in the doc as well.
Also, I'm sad to know AVERAGE booking is not available yet :-( On Sun, Oct 30, 2016 at 6:31 PM, Martin Blais <[email protected]> wrote: > It has been a good 6 months since the last update. > Lots of important changes recently - in particular, moving to the newer > and better booking algorithm, requiring Python 3.5 and I've deprecated a > bunch of old deprecated stuff. It's possible you will experience > user-visible changes. > Full details below. > > > > 2016-10-30 > > - Merge the current version of the 'booking' branch and made the "FULL" > booking > algorithm the default one. > > * The "experiment_booking_algorithm" option has been renamed to > "booking_algorithm". Its default value is now "FULL"; everyone will > be > using the new booking code. If for some reason you have a > transitional > need to revert to the older booking method use this option. > > option "booking_algorithm" "SIMPLE" > > * The new booking algorithm is _MUCH_ more powerful than the one old. > I'll > describe it soon in a new document, but briefly: > > - It supports many more interpolation capabilities; try omitting > amounts > or currencies which you think it ought be able to figure out > automatically and you might find it does. > > - All positions carry a date. This means that if you have > transactions > that would merge because they weren't specified with a date, like > this > you may face a problem: > > 2016-01-15 * > Assets:Investments 70 UUG {10.00 USD} > ... > > 2016-02-16 * > Assets:Investments 25 UUG {10.00 USD} > ... > > 2016-03-30 * > Assets:Investments -80 UUG {10.00 USD} > ... > > Those used to resolve automatically because both augmenting lots > would > merge into a single one without a date. Now that all positions > have an > acquisition date, the -80 reducing posting is ambiguous, because it > matches against these: > > Assets:Investments 70 UUG {10.00 USD, 2016-01-15} > Assets:Investments 25 UUG {10.00 USD, 2016-02-16} > > Either you'll have to disambiguate, or let Beancount do so. One > way to > do this is to add more specific matching data, and split the > posting. > An even easier way to do this is to make the default booking > method of > this account to "FIFO" or "LIFO", which will automatically select > the > oldest (newest) postings without issuing a warning. > > If you find yourself in a bind, please email the mailing-list with > a > specific and as small as possible data input that reproduces the > problem. > > - Removed parsing of the pipe (|) symbol as a separator for strings on > transaction description lines. This was vestigial from the days long > long > ago when Beancount was attempting to be compatible with Ledger syntax; > this > is wholly unnecessary how. > > You will have to convert transactions like this: > > 2015-08-25 * "Tunnel" | "Coffees" > Expenses:Food:Coffee 15 CAD > Assets:Cash > > Into transactions like this: > > 2015-08-25 * "Tunnel" "Coffees" > Expenses:Food:Coffee 15 CAD > Assets:Cash > > This can be done using a sed script, or as a macro in your favorite > text editor. > If you need a temporary respite, you can use the following option, > which > will replace all the error messages by a single warning: > > option "allow_pipe_separator" "TRUE" > > Support for this option will be removed eventually. > > > - Moved the {{...}}} total cost syntax from a legacy support with only > "/" > separator and a date, no label, to full support. Previously, only > these syntaxes > were supported for total cost: > > Assets:Account 100 MSFT {{2000.00 USD}} > Assets:Account 100 MSFT {{2000.00 USD / 2016-10-30}} > > Changes: > > * The "/" separator support has been removed, you must now use a > comma, like > for the {...} full booking syntax. > > * The same syntax as for the regular {...} compound cost is supported; > however, specifying a compound amount will raise an error, and the > per-unit cost is interpreted as a total cost. > > For example, this: > > Assets:Account 100 MSFT {{2000.00 USD}} > > is interpreted as: > > Assets:Account 100 MSFT {0 # 2000.00 USD} > > which is equivalent to: > > Assets:Account 100 MSFT {20.00 USD} > > And the following will raise an error, it is not legal syntax: > > Assets:Account 100 MSFT {{0 # 2000.00 USD}} > > Finally, specifying a date and a label can be done, just like for > {...}: > > Assets:Account 100 MSFT {{2000.00 USD, 2011-10-30, "ref32"}} > > The intention here is to deprecate the older limited syntax while > keeping > support for total cost input using {{...}}. Based on previous > questions, I > suspect that many users are still using {{...}} instead of {{# ...}}, > so I > won't remove it just yet. > > - Removed very old compatibility code in beancount.core.amount that lived > since I cleaned up the definition of the D() constructor and moved it > to > beancount.core.number. > > - Removed built-in aliases for auto_accounts and implicit_prices > modules, from > beancount.ops to beancount.plugins. > > - Removed deprecated option "default_tolerance" (which had been renamed > to > "inferred_tolerance_default"). > > - Removed deprecated option "tolerance". > > - Remove deprecated use of 'plugin' as an option. The directive ought to > be > the only way. > > - Removed deprecated option "experiment_explicit_tolerances". > > - Renamed "experiment_booking_algorithm" to "booking_algorithm", and > changed > the default value to "FULL". The old "SIMPLE" algorithm will be > deprecated > eventually. > > - Removed old code that would warn if the user set account_rounding to a > full > account name; it just needs be a subaccount. > > - Commit to Python 3.5. 3.5.0 released more than a year ago. You need to > update or stick with the 2.0b13 release. > > - Remove old code to dispatch between an external Enum class and the > built-in > one that's included in 3.5 (commit to 3.5). > > - Added type annotations to beancount.core.data. This will grow more over > time. (I also need to add rules to use those.) > > > 2016-10-23 > > - Modified parsing of transaction strings and links and tags. I did this > to > factor out the parsing of tags and links in order to add it to other > transactions and minimize grammar code. There is user-visible impact > to this > change: all of a transaction's strings must appear before tags or > links. You > cannot place tags or links before or in-between strings anymore. This > is a > reasonable restriction and I doubt anybody did anyway. > > - Added support for tags and links to the Document directive. This works > just > as for Transactions, e.g. > > 2016-10-23 document Assets:Checking "/statements/oct.pdf" #banking > ^9fed846348c2 > > The current tags on the tag stack are also included, like for > transactions. > > > 2016-10-10 > > - Removed "experiment_explicit_tolerances" flag; made this supported by > default permanently. > > > 2016-10-08 > > - Documented a subtle problem reported about balances against inventory > of > mixed at-cost and no-cost, added more context to the error and a unit > test > for balance checks with mixed at-cost and no-cost positions. > > - Fixed a bug in the full booking matching code whereby an existing > position > not held at cost would trip up the matching code. (Credits to Son > Nguyen for > finding and reporting.) > > - During ingest, when multiple extractors match a particular file, > select the > first matching account instead of raising an error. > > - Fixed accidentally colliding key-binding for invoking bean-price from > Emacs, > to C-c p. > > > 2016-09-25 > > - Fixed typo in grammar that was parsing a transaction flag as '#' into > '*'. > > - Added "cost_date" and "cost_label" columns in the SQL shell. > > > 2016-09-04 > > - Implemented new "run" shell command that allows you to run a named > query. > For example, if an input file has a Query directive like this: > > 2016-09-01 query "home" "select account, sum(position) where account > ~ 'Home' group b$ > > And you input this command: > > run home > > The entries will be closed at tue query date (2016-09-01) and the query > automatically run. If you want to always include all the transactions, > set a > date far into the future. > > This works from the command-line and from the shell, equally. You can > also > run all the queries in an input file, like this: > > run * > > To view the list of available queries, just type the "run" command > without > an argument. > > > 2016-08-21 > > - Fixed a SQL shell bug with extracting UNITS() from a Position column. > > - Finally properly fixed the timezone of the Google Finance price > fetcher. > (It was done incorrectly and I found my unit tests failing whenever I > would > work on Beancount from another timezone.) > > > 2016-08-17 > > - Added beancount.core.amount.abs() function for absolute values of > Amount. > > - Added rendering of links to beancount.plugins.split_expenses plugin, > and > made it quiet by default. Added --output-stdout option to make it do > that > again. > > - Created a new experimental script that attempts to match postings > between > two ledgers. A pair of (filename, account-regexp) is provided for > each, and > then the amounts and links are compared and paired up, and remaining > unmatched postings are printed out. This is very useful to reconcile > trip > expenses between a personal ledger and the ledger for a trip or > external > project where some of the expenses are paid from the personal ledger as > contributions to the trip or project. > > - Made the upload-csv-to-google-sheet script use the basename (without > extension) of the given filenames as sheet names instead of "Sheet X" > numbers. (Note: This entire thing be rewritten using the new Google > Sheets > API which won't require gdata and which will be more stable.) > > - Added a quick implementation of b.q.query_render.render_csv() > and refactored b.q.query_render.render_text() to use common code. Both > functions now have a new 'expand' option which will create multiple > rows for > cells which return a list of strings. The new default behavior is to > concatenate the strings with commas. > > The shell (bean-query) has a new 'expand' shell variable to match. > Also, the > -f/--format option of it is now supported, and in interactive mode it > sets > the default value for 'expand'. > > - Since output to CSV is now implemented, I enabled the --output-csv > option of > b.p.split_expenses as well. > > > 2016-08-14 > > - Fixed bug in beancount.utils.file_type for Python3, whereby magic now > returns > bytes, not a string > > 2016-08-04 > > - Made account name components allowed to have two characters (beyond the > first component). For instance, Assets:Investments:F is now a valid > account > name. > > > 2016-07-11 > > - Added parsing a transaction date column to metadata in the CSV > importer. > > > 2016-07-01 > > - bean-format now accepts input from stdin. > > > 2016-06-30 > > - Make it possible for a BALANCES statement to have a WHERE clause. > BALANCES > is now equivalent to > > SELECT account, sum(position) FROM ... WHERE ... GROUP BY account > > Inching one step closer to removing that WHERE clause eventually. > > > 2016-06-18 > > - Added support for columns AMOUNT_DEBIT and AMOUNT_CREDIT in the csv > importer. > > - Merged ongoing progress from 'booking' branch; if you don't use any of > the > in-progress new booking methods, this should not have any effect. > > However, the 'booking_method has changed names and you may have to > adjust > its values if you've been tinkering with it: > > * 'booking_method' used to take on value "SIMPLE" or "FULL" and > dispatch > betwee the current (old) booking implementation and the new one > which will > support many other booking methods. This option is now named > 'experiment_booking_algorithm' (a new option) and takes on the same > values. The default value is "SIMPLE", which keeps the behavior > unchanged. > Use "FULL" if you want to test out the new booking algorithm: the new > algorithms should work except for the AVERAGE cost method. WARNING: > This > is still under flux and considered unreleased. > > " 'booking_method': This is a different option now, and takes the > value of > any one of the booking methods visible here: > https://bitbucket.org/blais/beancount/src/ > 7b9e00e6e19136f680befc1d4322f47cdc294cf6/sr$ > > > 2016-06-13 > > - Ported beancount.docs.download_docs to the v3 drive API and the use of > a > service account instead of a full access to a user account. Service > accounts > are easier to configure and work with. > > - Ported beancount.docs.upload_options to the v3 drive API. > > > 2016-06-05 > > - Added a new column to the posting context: "other_accounts" is a set of > account strings for the other postings of the transaction. > > - Added a new JOINSTR() generic function to the shell, to reduce the > repeated > column above to a single comma-separated string. > > > 2016-05-28 > > - Made various changes to the shell in order to make it possible to > convert to > the market value: > > * CONVERT() has been converted to convert to the market value, when > applied > to a position held at cost. However, it still requires a target > currency > and it may fail if you apply it to a position for which the cost > currency > differs from that of the target currency, i.e., it will not > automatically > chase down the transitive conversions toward the desired target > currency. > > * A new function VALUE() has been added, which is similar to CONVERT() > but > which when applied to a position held at cost converts to the cost > currency at the current price. It's similar to CONVERT() but you > don't > have to provide the target currency. And similarly to CONVERT(), you > may > provide a particular date, and if at that date there is no available > conversion rate in the prices database, the conversion will fail > analogously and spit out an error message. > > * PRICE() has been renamed to GETPRICE() to avoid some confusion. > PRICE() > does not convert, it merely gets the price from the price database. > GETPRICE() is thus a more appropriate name. > > > 2016-05-22 > > - Created the beancount.plugins.mark_unverified plugin as a response to > this > thread: https://groups.google.com/d/msg/beancount/hQoOSJE7qNo/-E__ > v-XhEgAJ > > This plugin makes it possible to compute the balances as of the last > Balance > directive of each account by adding metadata to postings which appear > after > that account's last balance. You'd filter out unverified postings with > an SQL > query on meta-data like this: > > SELECT account, sum(position) > WHERE NOT meta('unverified') > GROUP BY 1 > ORDER BY 1 > > Note that the resulting balanaces almost certainly don't sum up to > zero, > because of differing dates of the Balance directives. Also, note that > accounts without a single Balance directive will not have their > postings > marked at all. > > > 2016-05-08 > > - In the output of bean-price, sort prices exclusively by currency, > regardless > of date. This is because on weekends you'll tend to get slightly > varying > dates, and manual/occasional price updates are usually put in "chunks" > in a > file section. It's easier to visually find the price by scanning by > currency > order. > > > 2016-05-05 > > - Added a shell function to filter a set of strings (for tags or links) > with a > regular expression and return the first one. This way you can extract > some > of the tags or links with a pattern as columns. > > > 2016-05-01 > > - Fixed issue #124, a leak in the grammar parser which would leak the > entire > set of entries on every instance of parse. > > - Fixed some warnings about unclosed files which occur only on default > python > branch. > > > 2016-04-29 > > - Don't just log an extraction error to logging.error() in the regression > tests; let the exception through. This makes building and debugging > importers much easier. > > - Created an experimental script to extract date/location information > required > for a naturalization application. > > 2016-04-24 > > - Rewrote the section on "Core Data Structures" of the Design Doc, to > reflect > the newer internals of Beancount, those which have been merged from the > ongoing 'booking' branch. > See http://furius.ca/beancount/doc/design-doc. > > > 2016-04-23 > > - Added beancount.ingest.importers.fileonly, a simplistic importer > which can > be instantiated to just file away some downloads without extracting > anything > from them. > > > 2016-04-18 > > - In bean-doctor context, all the accounts should appear in the > post-transaction balances, even if their balance is empty. This had > been > fixed for the pre-transaction balances but I stumbled upon a case of > post-transaction balances not showing an empty account. > > > 2016-04-17 > > - Added support for the BEANCOUNT_LOAD_CACHE_FILENAME environment > variable, > which can be set in order to overridde the location of the pickle > cache. > This variable may contain the string "{filename}", and if it is > present the > basename of the beancount file will be spliced in. This can be useful > if, > for example, you are reading your Beancount input file from a read-only > filesystem and still want to use the cache by specifying an alternative > filename. > > - Fixed a bug when reading from a read-only filesystem, the loader cache > would > fail to be removed and raise an exception. > > - Fixed an important bug in beancount.ingest whereby instances of > b.i.cache.FileMemo weren't being shared as widely as they could have > been. > FileMemo instances now have to be created via b.i.cache.get_file() and > FileMemo is now a hidden class _FileMemo. See this discussion for more > details: > https://groups.google.com/d/msg/beancount/b-ZqYqag7AU/N-AaGSZOAwAJ > > > 2016-04-10 > > - Create a new 'beancount.plugins.fix_payees' plugin that allows the > user to > provide a set of rules to rewrite and clean up the names of the payees > from > the Beancount file. > > While it would be ideal to do this at import time, in practice there > are a > lot of downloaded files' "memo" fields which get set as the payee > automatically. These "memos" aren't very clean and readable strings: > they > sometimes contain numbers, partial city names, and other junk. In > order to > produce clean reports where one might want to aggregate by payee, it's > useful to rewrite the payees to make them nice and clean. You can use > this > plugin to do that. > > > - Made the Custom directive accept ACCOUNT tokens. You can now do > something > like this: > > 2014-06-20 custom "budget" Assets:Account2 "balance < 200.00 USD" > > Prior to this, the account name had to be represented as a string. > (Users of > the burgeoning Custom directive not like this very much.) > > Note that account names and regular strings are both output from the > parser > as 'str' objects, so in order to make this possible, I had to change > the > representation of the values to a pair of (value, dtype). Account > names have > a 'dtype' of beancount.core.account.TYPE. (Eventually I will probably > want > to make account strings represented with their own custom datatypes, > but > that will require a fair amount of work on the codebase; this > substitute is > fine for now, as there are no other places where such a disambiguation > is > necessary.) > > > 2016-04-04 > > - Added a test to make sure that building a source dist includes the > header > files and C implementation files. > > - Sort Document directives after Transactions when they occur on the > same day. > The reason for this is that statements typically include transactions > on > that final day, and we'd like for them to occur at the right place in > the > journals. See originating thread: > https://groups.google.com/d/msg/beancount/3ItOYS9DJx4/VyTJB3ssBQAJ > > > 2016-04-03 > > - Fixed a bug in "bean-doctor context" which wouldn't report the full > set of > accounts when an account's before-balance would be empty. > > - Fixed a bug in "bean-doctor linked" whereby the balances wouldn't get > printed out if there was no link. Print the balances before and after; > in > fact, maybe this command should be renamed to "balances" and have an > option > to support links. > > > 2016-03-31 > > - Added TODAY() function to extract today's date. Not sure how useful > it'll > be. > > > 2016-03-27 > > - Added a PRICE() function to extract entries from the price database. > You can > now query for "SELECT PRICE('USD', 'CAD', date)", for example. > > - Added a boolean renderer for the SQL shell. > > > 2016-03-26 > > - Added an optional date argument to convert, so you can provide the > requested > date in a SQL query, as in "CONVERT(sum(position), "CAD", 2015-06-01)". > > - Improved the rendering of the context, removed the weird comments it > used to > render with, render an account name if the inventory is empty. > > - Ignore .DS_Store files from the file finder in ingest. > > > 2016-03-22 > > - Support multiple accounts in same ofx file. See > https://bitbucket.org/blais/beancount/pull-requests/10/. I applied > this > one-liner: issue a balance for each statement, use the maximum date > for the > file_date() method, and added unit tests for this. > > > 2016-03-21 > > - Fixed bug in beancount.ingest.file whereby two files being filed to > the same > destination filename would clobber each other; with this fix, instead, > an > error will be generated. > > - Create an example of an importer just filing a PDF file using the > external > tool PDFminer2. If the tool isn't installed, nothing should "break" > (but the > importer won't really do its job). The tests should be skipped > automatically, and the importing will just ignore the PDF file to be > filed. > > > > > -- > You received this message because you are subscribed to the Google Groups > "Beancount" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/beancount/CAK21%2BhNTCgSibc5xVYzCkPyknz%3DFmN3cTM4VEawEEmwxH2zE6g% > 40mail.gmail.com > <https://groups.google.com/d/msgid/beancount/CAK21%2BhNTCgSibc5xVYzCkPyknz%3DFmN3cTM4VEawEEmwxH2zE6g%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- Yuchen Ying http://about.me/yegle -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAFL5w3UCJTuT0PqzYQM-STwAfOPk5_LHFOOdDhtHDwzKMiEM4g%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
