On Fri, Jan 16, 2015 at 11:01 PM, Red Street <[email protected]> wrote:
> Excellent. Thanks much, Martin, and Simon. Precisely what I was looking > for. I'm not setup with hledger currently, though I've tried it in the > past, so I'll start with Martin's solution for now. > > Martin: from just looking at the code, it seems like I don't even need to > run ledger or beancount, since this simply parses the journal file > directly? > This is a bit of an oblique question; let me clarify what I mean by this: it uses Beancount. It uses its loader to parse and load the input file (where "parse" means to produce data structures that match 1:1 to the input, and "load" means further process this list with plugins which may themselves generate more directives or modify those, and then validates various things). This forms the entire "input" part of Beancount, including all its semantics; it always begins with load_file(). But essentially, that _is_ most of what Beancount is. Everything else is just different tools built on top of this produced list to extract different aggregations and filters (bean-web, bean-report, bean-query, or a custom script). In this case, this is a custom script, it runs through the loaded data structure (a list of directive objects) to filter out a subset of transactions and compute returns from them: https://bitbucket.org/blais/beancount/src/9aa7842c801b02a5ddc70177ca69927310ea8565/src/python/beancount/projects/returns.py?at=default#cl-850 About parse vs. "load": https://bitbucket.org/blais/beancount/src/9aa7842c801b02a5ddc70177ca69927310ea8565/src/python/beancount/loader.py?at=default#cl-107 Also again, looking at your code, it builds up a pricedb as it goes along > if one wasn't supplied, but does it end up taking the last known value for > a stock on the final day of the period under consideration? Can I give it a > pricedb of just the last day in the period, since in theory, that's all is > needed? > The price database is always automatically created from a list of loaded entries: https://bitbucket.org/blais/beancount/src/9aa7842c801b02a5ddc70177ca69927310ea8565/src/python/beancount/projects/returns.py?at=default#cl-627 https://bitbucket.org/blais/beancount/src/9aa7842c801b02a5ddc70177ca69927310ea8565/src/python/beancount/ops/prices.py?at=default#cl-57 You don't need to do anything special to get a price map in the code. Price entries are created either implicitly as a result of postings with cost ({...}) or with price conversions (@ ...), or explicitly as directives, e.g. "2015-01-06 price HOOLI 505.23 USD". You can use "bean-report <filename> print" to view the list of generated price directives after loading. To answer your question about prices and dates: the price database will interpolate from the last seen price. It would be wise to add price points on the dates where you have external flows, for all the commodities you're holding at that date, as well as the dates where you intend to start/stop calculating the returns. (One further improvement I want to add is an automated warning when prices are interpolated too far from their sample points, to let the user know which price points should be manually inserted in order to compute the returns most precisely. Interpolation does an okay job to get a good approximation, but it would be nice if the script would help us adding the missing prices; I thought it could produce a list of Price directives with zero values and another script could eventually fetch all those missing prices.) Question about granularity: if I have n different commodities I bought, is > it possible to compute the return for each commodity separately? And if I > have n different investment accounts (say, a retirement account, and a > non-retirement account), should I run this on these accounts separately if > I want to know their individual returns? > Yes, and yes, but with a caveat. You have to create clearly segregated accounts in order to achieve this level of separation, but IMO that is congruent with good chart-of-account design. For example, if you wanted to compute the returns for a particular commodity with dividends (and include the dividends in its return), you would have to either (a) have an income account defined specifically for that commodity,e.g. Income:US:Ameritrade:Dividends:HOOLI, or (b) if you use a more general account to book that income, e.g., Income:US:Ameritrade:Dividends for all dividends from an Ameritrade account, then you'd have to filter and compute only off of entries that are pertinent to your commodity, that is, that include only those dividends received as a consequence of holding a position in that commodity and not those that pertain to dividends received for other commodities (this isn't entirely obvious to import, since unlike capital gains, dividends will typically only have postings on the income account to the cash accounts, so it's usually something in the memo ("description") string of your imported transaction that tells you for which commodity the dividend is given, there are no commodities involves in the postings, just cash). How you create or import these is up-to-you. In any case, you have to find some way to isolate those entries (the regular expression method I provide is one such way). The hard part of designing a scheme for computing returns was to figure out a way to categorize account names in such a way that the script could (a) compute the value of the portfolio (the sum of "value" accounts), (b) incur trading costs (expenses) and income without causing external flows (to/from "internal" account), and (c) correctly identify external flows (to/from "external" accounts) in order to value the portfolio before and after that boundary. What I've done so far is to work this out by classifying each account, and figured out a way to segregate the set of all accounts involved in a subset of entries into these categories. This is described here: https://bitbucket.org/blais/beancount/src/9aa7842c801b02a5ddc70177ca69927310ea8565/src/python/beancount/projects/returns.py?at=default#cl-7 The downside of this is that you kind-of need to understand this to get it working. But I believe this fully generalizes to all possible cases of returns calculations you might want to design. This was my goal: support all my accounts types. I've managed to get this working with most of them (I still have some remaining kinks in one of them). In order to use this, your first task is to determine the list of all accounts affected by the transactions you will select, and which category each of them are supposed to belong to. I provided a way to specify this with two regular expressions (or if you want you can otherwise write code to be more explicit) . Then you run the script with logging turned on and to verify the account names were correctly categorized, and then you will see all the returns segments and included transactions, that you should cross-check in order to make sure it's doing what you expect it to. The tests offer various illustrative examples: https://bitbucket.org/blais/beancount/src/9aa7842c801b02a5ddc70177ca69927310ea8565/src/python/beancount/projects/returns_test.py?at=default I'm sorry for the long and daunting explanation; this isn't a trivial problem to solve, at least in the general form. I would recommend reading the docstring, the tests, then starting with a small subset of your own transactions and using the --verbose flag and inspecting what it's doing. If you have ideas about how to make setup and debugging easier, let me know. I think this could be made a bit easier to setup. My goal was to get to the point where I could automatically compute my portofolio returns together and by account, over arbitrary periods of time (e.g. last month, last three months, last year, last two years, calendar year). I hope this helps. Feel free to come back with more specific questions once you get going. Cheers, > > Thanks much again! > > -- > > --- > You received this message because you are subscribed to the Google Groups > "Ledger" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "Ledger" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
