Nice approach Red. I use a lot of the same institutions and have been working with beancount-import. For example see Schwab CSV importe <https://github.com/jbms/beancount-import/blob/master/beancount_import/source/schwab_csv.py>r with support for brokerage, options, and banking. I'm evaluating switching to the native beancount importer protocol vs beancount-import so this is a great example to draw on.
beancount-import to me has these pros: - I like the web UI for interactive import and for its ability to propose joins between different sources generating two different halves of a transaction. - Nice deduplication support - Ability to see transactions that are uncleared - that is those you've imported the other half of but the 'authoritative' source hasn't seen it yet. Downsides: - The ML prediction algorithm is overly broad in what it considers for training examples, leading to poor results <https://github.com/jbms/beancount-import/issues/92>. Hopefully smart_importer is better about this, I plan to evaluate it. - It's built around the idea that you'll use web scrapers / tools to download all the files into the correct directories. Scrapers are the most brittle part of my infra so I manually download many of my accounts instead. For that, the beancount importer tooling is a great fit. On Sun, Feb 14, 2021 at 6:09 PM [email protected] <[email protected]> wrote: > Neither have been tested with options trades though. If you use them, I'd > be curious to know what your experience is. > > The importers are separated into three parts: > > 1. file format reader (reusable) > 2. transaction builder (reusable) > 3. institution-specific declarations and code (minimal, institution > specific) > > > > On Sunday, February 14, 2021 at 6:07:09 PM UTC-8 [email protected] wrote: > >> Importers: >> >> https://github.com/redstreet/beancount_reds_importers >> >> >> https://github.com/redstreet/beancount_reds_importers/tree/master/beancount_reds_importers/vanguard >> (well tested against several hundred files) >> >> https://github.com/redstreet/beancount_reds_importers/tree/master/beancount_reds_importers/schwab_csv >> (newer, tested against ~5 files) >> >> >> >> >> On Sunday, February 14, 2021 at 7:45:55 AM UTC-8 Rahul Kuchhal wrote: >> >>> Thank you! >>> >>> I see that adding new commodities is the recommended way then. >>> >>> Now on to finding Vanguard and Schwab importers or adopting your >>> Ameritrade importer for them. >>> >>> On Saturday, February 13, 2021 at 5:13:01 PM UTC-5 [email protected] wrote: >>> >>>> Good point about the price directive. I agree with your recommendation. >>>> Seems like the best way to handle it is to fix any reporting tooling that >>>> have problems with the flood of commodity symbols to aggregate or page >>>> their output. >>>> >>>> On Sat, Feb 13, 2021, 14:03 Martin Blais <[email protected]> wrote: >>>> >>>>> On Sat, Feb 13, 2021 at 1:34 PM Rahul Kuchhal <[email protected]> >>>>> wrote: >>>>> >>>>>> I am also trying to figure out the best way to represent Sold options >>>>>> that open new positions. >>>>>> >>>>>> The suggested approach below works (kind of). >>>>>> >>>>>> 2020-12-28 * "Sell March 1 PUT 200 on HOOL" >>>>>> Assets:Brokerage:Taxable:HOOL:Options -1 HOOL_PUT_200_2021_03_01 >>>>>> {1000 USD} >>>>>> Assets:Brokerage:Taxable:Cash 1000 USD >>>>>> >>>>>> Seems like I would need to add a new currency to HOOL:Options account >>>>>> each time I open a new position. Overtime I suspect this may result in >>>>>> hundreds (or even thousands) of currencies attached to a single account. >>>>>> >>>>> >>>>> That's what I do. My importer creates them automatically from the TOS >>>>> log. >>>>> Here: >>>>> >>>>> https://github.com/blais/ameritrade/blob/master/examples/ameritrade2beancount.py >>>>> >>>>> I'm doing vertical and calendar spreads, collars, covered calls, and >>>>> more. It all works great with Beancount. >>>>> I can give a little demo of how to use bean-doctor to interactively >>>>> compute the P/L of an isolated sequence of trades if anyone cares. >>>>> >>>>> The importer above groups the trades by underlying, and typically >>>>> these are positions being rolled, but not always. I may initiate multiple >>>>> separate trades with distinct history in the same underlying (either >>>>> closing and reopening, or over different months). >>>>> >>>>> One of the issues is that "regular" activity - stock purchases and >>>>> sales, or short positions, as well as longer-term hedging on LEAPs, all >>>>> happen in the same account. I consider these logically distinct from the >>>>> premium / vol trading I do, so I want to separate them out. Here's how I >>>>> do >>>>> this: when I import, I keep the active options traffic segregated to an >>>>> include file, one per week. I manually move the long term activity to my >>>>> main Beancount file, where it will remain forever. All the transactions >>>>> are >>>>> tagged with TD's unique transaction ids, and when the importer runs it >>>>> automatically ignores anything that's been imported before, here: >>>>> >>>>> https://github.com/blais/ameritrade/blob/master/examples/ameritrade2beancount.py#L912 >>>>> >>>>> I have been keeping each week in a separate file, with custom scripts >>>>> to compute realized and unrealized P/L by week. >>>>> However, as I've increased my volume and am holding positions over a >>>>> longer time, I'm finding I need something more sophisticated. >>>>> So I've started slowly moving away from using Beancount for that >>>>> particular activity, and have been building code that can automatically >>>>> untangle each trade directly from the TOS log, following it through >>>>> multiple rolls over time, and compute the total P/L once all the related >>>>> legs are closed out. It's nice as I can do per-trade analysis on spreads >>>>> and all their history of managing them that follows over multiple weeks, >>>>> as >>>>> a single "trade". >>>>> >>>>> As far as the main ledger goes, I think I'll probably end up >>>>> summarizing the impact of realized P/L on a weekly basis through a single >>>>> transaction in my main Ledger file. There's no reason I'd need to keep all >>>>> that traffic in my main file, even if I'd still use Beancount for it >>>>> (which >>>>> I'm not entirely sure I will for that activity). >>>>> >>>>> >>>>> >>>>> 1) Is there any downside of attaching so many currencies to a single >>>>>> account? >>>>>> >>>>> >>>>> No. I do think that we may want to eventually add a regexp type of >>>>> commodity that can cover them all though. >>>>> You shouldn't have to insert them manually, have your importer do that. >>>>> So far it works well for me. >>>>> >>>>> >>>>> >>>>>> 2) Is there a way to represent these transactions using the same >>>>>> currency over and over (based on the cost basis and date of options) >>>>>> >>>>>> 2020-12-28 * "STO March 1 PUT 200 on HOOL" >>>>>> Assets:Brokerage:Taxable:HOOL:Options -1 Options {1000 USD} >>>>>> Assets:Brokerage:Taxable:Cash 1000 USD >>>>>> >>>>>> 2020-12-30 * "BTC March 1 PUT 200 on HOOL" >>>>>> Assets:Brokerage:Taxable:HOOL:Options 1 Options {1000 USD} @ 990 >>>>>> USD >>>>>> Assets:Brokerage:Taxable:Cash -990 USD >>>>>> Income:PnL >>>>>> >>>>>> >>>>> I wouldn't do that. Part of the point of using Beancount is to figure >>>>> out what your inventory is and different options have different prices >>>>> over >>>>> time (I use the Price directive, works well). >>>>> Besides it would have to be OPTIONS. >>>>> >>>>> >>>>> On Tuesday, December 29, 2020 at 5:11:26 PM UTC-5 Rajath wrote: >>>>>> >>>>>>> Two good approaches. The one where each option symbol is >>>>>>> commoditized more inline with how stocks are managed. But the second >>>>>>> approach is also very interesting because it accounts for assignment >>>>>>> risk. >>>>>>> My brokerage also holds the collateral from available cash when I sell a >>>>>>> put option, so maybe I can find a way to represent that too (which says >>>>>>> exactly how much cash I have on hand to trade). >>>>>>> >>>>>>> I will play around with this! >>>>>>> >>>>>>> On Tuesday, December 29, 2020 at 1:16:10 PM UTC-8 [email protected] >>>>>>> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I count only the money in my account and control the sum of >>>>>>>> possible assignments. I don't have an "entity" option. >>>>>>>> >>>>>>>> For the options, I sell puts of stocks I want to buy and I have the >>>>>>>> money for doing it. So if it's assigned, no problem ;-) >>>>>>>> >>>>>>>> On Tue, Dec 29, 2020 at 6:47 PM Martin Blais <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> On Tue, Dec 29, 2020 at 11:24 AM Tuno Tunante <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> What I'm doing (I did the last yesterday) is like that: >>>>>>>>>> >>>>>>>>>> 2020-12-28 * "TD" "Sell Put MO 15 JAN 2021 Strike 40 @0.40" # >>>>>>>>>> Option #Put #MO #Sell >>>>>>>>>> Income:Options:TD -40.00 USD >>>>>>>>>> Expenses:Comission:TD 0.67 USD >>>>>>>>>> Assets:Account:TD 39.33 USD >>>>>>>>>> Assets:Options 4000.00 USD >>>>>>>>>> Liabilities:Opctions >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> Where's the option? >>>>>>>>> >>>>>>>>> >>>>>>>>> I do 6 or 7 per month. I use the Assets:Options and >>>>>>>>>> Liabilities:Options in order to have a complete view about the money >>>>>>>>>> I must >>>>>>>>>> reserve in case I'm assigned. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Rule #1 of options trading: never get assigned! ;-) >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> Before each put, I check that Liabilities:Options is less that >>>>>>>>>> the money in Assets:account:TD >>>>>>>>>> >>>>>>>>>> Regards. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Dec 29, 2020 at 6:18 AM Rajath Agasthya < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Yeah, I figured each option could be a new commodity. I don't do >>>>>>>>>>> a lot of option trades, so I'm okay with having more commodities >>>>>>>>>>> until >>>>>>>>>>> something better turns up. >>>>>>>>>>> >>>>>>>>>>> The thing I'm not sure of is how to represent a trade where I'm >>>>>>>>>>> getting a credit (from selling a put or even a short sale of a >>>>>>>>>>> stock). I >>>>>>>>>>> can't quite seem to figure out the matching posting for this >>>>>>>>>>> transaction if >>>>>>>>>>> I represent receiving credit as a posting into a cash account such >>>>>>>>>>> as >>>>>>>>>>> Assets:US:Schwab:Cash. Any thoughts on how to do this or an >>>>>>>>>>> example you can share? >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> On Monday, December 28, 2020 at 5:04:37 PM UTC-8 [email protected] >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> On Mon, Dec 28, 2020 at 5:47 PM Rajath Agasthya < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> First of all, thanks to all the contributors (especially >>>>>>>>>>>>> Martin) for this wonderful tool! I'm new to double entry >>>>>>>>>>>>> accounting and >>>>>>>>>>>>> Beancount, but I'm blown away at how simple yet powerful this >>>>>>>>>>>>> tool is, >>>>>>>>>>>>> especially when combined with Fava. >>>>>>>>>>>>> >>>>>>>>>>>>> I've been reading docs and I just started setting up my >>>>>>>>>>>>> beancount file. But as I setup my investment account, I'm a >>>>>>>>>>>>> little bit lost >>>>>>>>>>>>> as to how I should manage option trades. The trades where there >>>>>>>>>>>>> is a net >>>>>>>>>>>>> debit is straightforward (I think), but I'm not sure how to enter >>>>>>>>>>>>> trades >>>>>>>>>>>>> like selling a covered call or a put or a spread/multi-leg option >>>>>>>>>>>>> that >>>>>>>>>>>>> result in a credit. >>>>>>>>>>>>> >>>>>>>>>>>>> Are there some examples I can refer to on how to manage option >>>>>>>>>>>>> trades? The docs mention options trading as a future topic to be >>>>>>>>>>>>> tackled >>>>>>>>>>>>> and I unfortunately couldn't find anything in the mailing list >>>>>>>>>>>>> (perhaps I >>>>>>>>>>>>> didn't search properly). So if anyone can point me in the right >>>>>>>>>>>>> direction, >>>>>>>>>>>>> I would really appreciate it. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I do options trading, but I only record simple options, never >>>>>>>>>>>> combos like this. >>>>>>>>>>>> - The option can be encoded in the symbol name. This makes a >>>>>>>>>>>> lot of new commodities to declare and put out of commissions. I >>>>>>>>>>>> haven't yet >>>>>>>>>>>> figured out what needs to be done to make this a bit nicer (e.g. >>>>>>>>>>>> allow user >>>>>>>>>>>> to define a commodity regex/pattern for these). >>>>>>>>>>>> - For stable asset types, I use leaf subaccounts; for options I >>>>>>>>>>>> don't (would be too many accounts, and it feels silly to create an >>>>>>>>>>>> account >>>>>>>>>>>> for a single position). >>>>>>>>>>>> - For options combinations/"strategies", if you have some way >>>>>>>>>>>> to trade all the legs instantly, you should be able to either (a) >>>>>>>>>>>> insert >>>>>>>>>>>> all the individual legs and their prices in a single transaction, >>>>>>>>>>>> or, if >>>>>>>>>>>> the product is already securitized by your broker, trade it like >>>>>>>>>>>> any other >>>>>>>>>>>> product (one leg with the derivative instrument). >>>>>>>>>>>> >>>>>>>>>>>> Hope this helps, you do have to get a bit creative if you trade >>>>>>>>>>>> options, but, well, you trade options, so you will be able to >>>>>>>>>>>> figure out >>>>>>>>>>>> something :-) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Rajath >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>> Google Groups "Beancount" group. >>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>> https://groups.google.com/d/msgid/beancount/51c57efc-8915-4edd-981a-cb997c1f143cn%40googlegroups.com >>>>>>>>>>>>> <https://groups.google.com/d/msgid/beancount/51c57efc-8915-4edd-981a-cb997c1f143cn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>> . >>>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "Beancount" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/beancount/b12e5c7d-f2d7-4b9c-8131-3cac25b17810n%40googlegroups.com >>>>>>>>>>> <https://groups.google.com/d/msgid/beancount/b12e5c7d-f2d7-4b9c-8131-3cac25b17810n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "Beancount" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/beancount/CAM6BLCwOUD47fZz6o0WroYjvPHFhLcHYfSft6bdwiDW1sC%2B%2BSw%40mail.gmail.com >>>>>>>>>> <https://groups.google.com/d/msgid/beancount/CAM6BLCwOUD47fZz6o0WroYjvPHFhLcHYfSft6bdwiDW1sC%2B%2BSw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "Beancount" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>>> >>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/beancount/CAK21%2BhMp0mVafPWRtdPYgFiQ1GmPLfpZgfaY_fdJJ_oZW74gZA%40mail.gmail.com >>>>>>>>> <https://groups.google.com/d/msgid/beancount/CAK21%2BhMp0mVafPWRtdPYgFiQ1GmPLfpZgfaY_fdJJ_oZW74gZA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Beancount" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/beancount/652e0445-c9a1-4b0b-9084-67672d3207cbn%40googlegroups.com >>>>>> <https://groups.google.com/d/msgid/beancount/652e0445-c9a1-4b0b-9084-67672d3207cbn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Beancount" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> >>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/beancount/CAK21%2BhO_34Q_5k63ZfzP-8B%3DY%3DVg9CwYVQ4U8iL_ob_76jZabQ%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/beancount/CAK21%2BhO_34Q_5k63ZfzP-8B%3DY%3DVg9CwYVQ4U8iL_ob_76jZabQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- > You received this message because you are subscribed to the Google Groups > "Beancount" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/beancount/80458186-ec8d-4797-895c-7f856752ae6an%40googlegroups.com > <https://groups.google.com/d/msgid/beancount/80458186-ec8d-4797-895c-7f856752ae6an%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CACGEkZtLfJtJK%3DUdx%3DSA90bThi15LsvoJ6mK%2BJ%3DYor7EnMjsRw%40mail.gmail.com.
