On Sun, Jun 18, 2017 at 7:22 PM, Dave The Happy Singer <
[email protected]> wrote:
> Hey Martin,
>
> Thanks again for the thorough response. I think I'm struggling to grasp
> this line from the docs:
>
> >The “FIFO” method automatically select the oldest of the matching lots
> up to the requested size of the reduction.
>
> In the case I posted, there is only one lot in the Hotwallet inventory,
> large enough to cover both the deposit to the Coldwallet and the fee. Even
> if I simplify the case to a single withdrawal with no fee:
>
> option "operating_currency" "AUD"
>
> 2014-01-01 open Equity:Opening-Balances
> 2014-01-01 open Assets:AU:Cash AUD
> 2014-01-01 open Expenses:Financial:TransactionFees:BTC BTC
> 2014-01-01 open Assets:AU:Hotwallet BTC "FIFO"
> 2014-01-01 open Assets:AU:Coldwallet BTC "FIFO"
>
> 2014-01-01 * "Initial cash balance"
> Assets:AU:Cash 1000.00 AUD
> Equity:Opening-Balances
>
> 2014-01-02 * "Initial hotwallet balance"
> Assets:AU:Hotwallet 1.00 BTC {1000 AUD}
> Assets:AU:Cash -1000.00 AUD
>
> 2014-01-03 * "Deposit Cold storage"
> Assets:AU:Hotwallet -0.050000000 BTC {}
> Assets:AU:Coldwallet 0.050000000 BTC {}
>
> This fails with an error, but the following tx works:
>
> 2014-01-03 * "Deposit Cold storage"
> Assets:AU:Hotwallet -0.050000000 BTC {AUD}
> Assets:AU:Coldwallet 0.050000000 BTC {}
>
> Given I've specified FIFO (and in my case there is only one lot in the
> Hotwallet inventory anyway), I can't quite get my head around why I can
> leave out the number but not the currency.
>
The answer is:
(a) because it's written to be general and support multiple currencies for
the cost basis so for augmenting postings it doesn't infer from prior
history which currency it could be (i.e., it supports you using a different
one),
or, if you prefer
(b) because inference is defined mainly "locally", that is, inspecting
information only from the transaction itself, in order to keep things
simple and understandable without having to refer to other parts in the
file; any remote transaction somewhere far in the file could have an impact
on the inventory presented as input to your transaction, and if I had
designed the interpolation to depend much on that inventory's state you
might find odd and surprising errors elsewhere when you add or remove a
transaction anywhere. That could create a confusing game of whack-a-mole.
The main effect of your currency addition on the reducing posting is that
it disambiguates the currency of the augmenting posting (it's able to infer
that it must be AUD since the only other posting is AUD).
Furthermore, I think it's important to realize that reducing postings are
treated very differently than augmenting postings. Perhaps I should go back
and document that better. Generally, an augmenting posting much provide
most of its information or it should otherwise be possible to infer it
using interpolation (omitting at most one number per currency group).
Part of what's going on here is that you're using the cost basis in a way
that is rather unusual. For stocks, for instance, it's never a problem to
insert the full cost basis on purchases, including the currency (well, you
wouldn't really have to since the other postings are cash and fees).
Transactions where all postings use the cost basis are pretty rare
normally, usually only for cost basis adjustments. Using them everywhere
like this might require a bit of enhancement to make those more convenient.
Also, qualitatively, I think you're looking at your particular use case and
thinking "this should be obvious" but not inspecting all the other possible
use cases it has to support. I also think that enhancing the inference code
to make use of incoming inventory contents might not be as confusing as I
fear it could be and we should experiment with that more.
However, I had a further surprise: the cost basis in the Coldwallet becomes
> 2014-01-03. I presumed that once a cost basis had been attached to a lot,
> it would be preserved unless overridden explicitly.
>
Well by removing it from the Hotwallet, you did the equivalent of "selling
it."
The second posting is like a purchase, so its cost basis inherits that of
the transfer transaction.
Again, this is a bit of an oddball case compared to a stock trading
scenario.
For cost basis adjustments - which also require the maintenance of the
original date - you can override the date manually. I think I must have
that example somewhere in my docs.
The issue here is that you're expecting it to treat a cost basis "transfer"
differently than it's designed for.
This is designed for stocks and bonds and stocks and suck.
To take another example: I have two stockbroking accounts. One has my
> margin loan so I normally buy shares with that account. However when I come
> to sell I often transfer them to another account which has lower brokerage.
> It's not relevant to my cost basis which account I hold the shares in; I
> can move them around as I choose but I'd need my cost basis to reflect the
> reality of when I bought and sold a specific share. I presumed that the
> <Position> objects created on the augmenting leg(s) would inherit the cost
> basis of the reducing leg(s) where FIFO is used and unless I explicitly
> reset the cost basis.
>
The Transaction objects and algorithms don't know anything about a
transfer. They just make sure the total cost basis balances.
If you want to maintain the lot date, you need to override it explicitly.
In my experience such transfers are relatively rare (and somewhat difficult
to initiate, as the brokers must send share certificates if I'm not
mistaken, it's not an every day transaction) and it's designed for the more
common case.
Finally, if you are doing day-trading (e.g. 1000 trades/day), Beancount
isn't really designed for that. It might work, but if I were to go back to
trading electronically I'd write something dedicated for that use case.
It's designed more for the individual "investor" in mind (e.g. 10-100
trades/month).
Effectively, it seems the safe way to proceed is to use STRICT and to
> specify the cost basis explicitly on each tx.
>
Every augmenting transaction must either be specified explicitly or be
inferable (one missing number can be inferred in each currency group).
> But as you noted in our previous thread, these txs are quite small and
> frequent. But that would also realistically mean running bean-doctor before
> every entry I write to establish what my inventory is, and rely on my not
> making mistakes. If I had to correct an historical tx, I'd have to update
> the cost basis of every subsequent tx.
>
> If this is true, I'd probably write a helper script to 'do FIFO' for me,
> but before I start going too far down that road it may be that I am still
> just a bit confused about how beancount's FIFO behaviour works.
>
The FIFO works fine AFAIK - that operates on the reducing transactions.
The question is how to specify the augmenting ones.
Any thoughts on this?
>
Is being restricted to a single augmenting postings per transaction (so its
cost can be inferred) and having to put in the currency not working?
>
> Thanks again,
> Dave
>
> On Fri, 16 Jun 2017 at 13:45 Martin Blais <[email protected]> wrote:
>
>> On Thu, Jun 15, 2017 at 11:17 PM, Dave The Happy Singer <
>> [email protected]> wrote:
>>
>>> Hi Martin et al,
>>>
>>> I'm still trying to understand how to reliably book transactions without
>>> explicitly specifying the cost basis on every leg (e.g. with FIFO booking)
>>>
>>> Here is a test case with everything explicit, that passes bean-check. I
>>> buy 1 BTC for 1000 AUD from an account initialised with a positive AUD
>>> balance. Then I transfer a sum to a cold wallet and back to the hot wallet,
>>> incurring a fee on each transaction which I book to a BTC-denominated
>>> expenses account.
>>>
>>> option "operating_currency" "AUD"
>>>
>>> 2014-01-01 open Equity:Opening-Balances
>>> 2014-01-01 open Assets:AU:Cash AUD
>>> 2014-01-01 open Expenses:Financial:TransactionFees:BTC BTC
>>> 2014-01-01 open Assets:AU:Hotwallet BTC "FIFO"
>>> 2014-01-01 open Assets:AU:Coldwallet BTC "FIFO"
>>>
>>> 2014-01-01 * "Initial cash balance"
>>> Assets:AU:Cash 1000.00 AUD
>>> Equity:Opening-Balances
>>>
>>> 2014-01-02 * "Initial hotwallet balance"
>>> Assets:AU:Hotwallet 1.00 BTC {1000 AUD}
>>> Assets:AU:Cash -1000.00 AUD
>>>
>>> 2014-01-03 * "Deposit Cold storage"
>>> Assets:AU:Hotwallet -0.050000000 BTC {1000
>>> AUD}
>>> Assets:AU:Coldwallet 0.049000000 BTC {1000
>>> AUD}
>>> Expenses:Financial:TransactionFees:BTC 0.001000000 BTC {1000
>>> AUD}
>>>
>>> 2014-01-04 * "Withdraw Cold storage"
>>> Assets:AU:Coldwallet -0.049000000 BTC {1000
>>> AUD}
>>> Assets:AU:Hotwallet 0.048000000 BTC {1000
>>> AUD}
>>> Expenses:Financial:TransactionFees:BTC 0.001000000 BTC {1000
>>> AUD}
>>>
>>
>> Side note: Your expenses should probably be best booked in AUD, not in
>> BTC.
>> In any case, it doesn't make much sense to store anything in an expense
>> account with a cost basis.
>>
>>
>> However, despite making my wallet accounts "FIFO", whenever I try to
>>> remove the explicit {1000 AUD} cost basis in my reducing postings, I start
>>> getting errors.
>>>
>>> For example, if I use {} in the first transaction:
>>>
>>> 2014-01-03 * "Deposit Cold storage"
>>> Assets:AU:Hotwallet -0.050000000 BTC {}
>>> Assets:AU:Coldwallet 0.049000000 BTC {}
>>> Expenses:Financial:TransactionFees:BTC 0.001000000 BTC {}
>>>
>>> I would be hoping that as FIFO is specified for the Coldwallet account,
>>> the lot is inferred.
>>>
>>
>> The reducing lot is inferred (the first one).
>>
>>
>> However, bean-check now complains: Failed to categorize posting.
>>>
>>
>> For the augmenting lots (the second and third one), there's no way to
>> infer what the cost basis should be, only you know that. e.g., you can use
>> this syntax to do cost basis adjustments.
>> Except, you may still be able to skip ONE number for each currency group.
>>
>> But... you also cannot omit too many currency groups. Consider the case
>> where you might use a different currency on the other leg (this has
>> happened to me before for a stock exchange between CAD and USD if I
>> recall). Anyhow, disambiguating the currency is easy; for example, this
>> works:
>>
>> 2014-01-03 * "Deposit Cold storage"
>> Assets:AU:Hotwallet -0.050000000 BTC {1000 AUD}
>> Assets:AU:Coldwallet 0.049000000 BTC {999 AUD}
>> Expenses:Financial:TransactionFees:BTC 0.001000000 BTC {}
>>
>>
>>
>> Furthermore, attempting to debug with bean-doctor doesn't quite work for
>>> me. The transaction in question starts at line 17. But beandoctor context
>>> <filename> 17 returns the context for the previous transaction at line 13.
>>>
>>
>> That's interesting; if there's no error, it works fine.
>> However, I think in this case, because this error was irrecoverable, the
>> transaction was skipped, and the behavior of the line-number selection is
>> to select the closest previous transaction. It does result in an odd
>> behavior.
>>
>> Maybe I should always create transactions even when there are errors.
>> https://bitbucket.org/blais/beancount/issues/169/consider-
>> always-creating-some-transaction
>>
>>
>> Which it correctly infers to:
>>
>> bean-doctor context smile2.beancount 19
>> Hash:b8a74ca220f3ebc6fce9da5d1ddd8738
>> Location: /home/blais/r/q/beancount-data/user/smile/smile2.beancount:17
>>
>> ------------ Balances before transaction
>>
>> Assets:AU:Hotwallet 1.00 BTC
>> {1000 AUD, 2014-01-02}
>>
>> Assets:AU:Coldwallet
>>
>> Expenses:Financial:TransactionFees:BTC
>>
>>
>> ------------ Transaction
>>
>> 2014-01-03 * "Deposit Cold storage"
>> Assets:AU:Hotwallet -0.050000000 BTC {1000 AUD,
>> 2014-01-02} ; -50.000000000 AUD
>> Assets:AU:Coldwallet 0.049000000 BTC {999 AUD,
>> 2014-01-03} ; 48.951000000 AUD
>> Expenses:Financial:TransactionFees:BTC 0.001000000 BTC {1049 AUD,
>> 2014-01-03} ; 1.049000000 AUD
>>
>>
>> Tolerances: BTC=5E-10
>>
>> ------------ Balances after transaction
>>
>> * Assets:AU:Hotwallet 0.950000000 BTC
>> {1000 AUD, 2014-01-02}
>>
>> * Assets:AU:Coldwallet 0.049000000 BTC
>> {999 AUD, 2014-01-03}
>>
>> * Expenses:Financial:TransactionFees:BTC 0.001000000 BTC
>> {1049 AUD, 2014-01-03}
>>
>>
>>
>>
>>> If I try amending the second transaction to use {} (instead of the first
>>> which I set back to {AUD 1000}), bean-check replies "Too many missing
>>> numbers for currency group 'AUD".
>>>
>>
>> How would you like Beancount to know which currency should be used?
>>
>> I've been thinking in the past that perhaps it should infer it from the
>> prior inventory--e.g., if there's a single (unambogious) cost currency
>> present in the ante-inventory's list of positions, fill that in
>> automatically-- but that would be more inference than it has done before. I
>> don't dislike the idea.
>> https://bitbucket.org/blais/beancount/issues/170/try-
>> inferring-currency-from-the-contents
>>
>>
>>
>>> And bean-doctor ... 13 shows the expected balances after the first
>>> transaction but beandoctor ... 17 doesn't show any opening balances even
>>> though it directly follows the tx at 13.
>>>
>>
>> Also probably the same issue, transaction might have been skipped due to
>> error.
>> Hmm I should improve that, I have seen that problem from time to time, it
>> is a bit of an annoying behavior.
>>
>>
>> Any idea what I may have done wrong here? I'm using beancount==2.0rc1 as
>>> installed by the latest package on PyPI. I've read the Inventory and
>>> Trading docs several times, but obviously I'm still learning so if I've
>>> wasted your time I apologise. Still excited by beancount!
>>>
>>
>> You're not wasting our time; please keep bringing up interesting corner
>> cases like this.
>> I'm not really interested in Bitcoins, but I like that it's dredging up
>> things at the edge of the booking code...
>> This is all good testing and discussion, if you have the patience.
>>
>>
>>
>> Dave
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Beancount" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To post to this group, send email to [email protected].
>>> To view this discussion on the web visit https://groups.google.com/d/
>>> msgid/beancount/d559299b-9f6e-45f6-bec4-996d2b52b4ef%40googlegroups.com
>>> <https://groups.google.com/d/msgid/beancount/d559299b-9f6e-45f6-bec4-996d2b52b4ef%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Beancount" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/
>> topic/beancount/bMKLxoWJb04/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> [email protected].
>> To post to this group, send email to [email protected].
>> To view this discussion on the web visit https://groups.google.com/d/
>> msgid/beancount/CAK21%2BhM3rQdqZ_k1LmY6XLWoGfgm0SpYMNicVHcUxzcO
>> fyv%2Bgg%40mail.gmail.com
>> <https://groups.google.com/d/msgid/beancount/CAK21%2BhM3rQdqZ_k1LmY6XLWoGfgm0SpYMNicVHcUxzcOfyv%2Bgg%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Beancount" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/beancount/CAHkYY6W-DdPB8n5WS7b%3DB4SiM%3DiOjy%
> 3D67JukaR6%3Dc03bikU5rg%40mail.gmail.com
> <https://groups.google.com/d/msgid/beancount/CAHkYY6W-DdPB8n5WS7b%3DB4SiM%3DiOjy%3D67JukaR6%3Dc03bikU5rg%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Beancount" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/beancount/CAK21%2BhMMTB_hDbxqJ1ipGMzFLn9raLy6JTzUFD_ksiEsvhh8aw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
