Let's profile further, and if needed, to write the parts most called in C.
On Sun, Oct 1, 2017 at 6:22 PM, <[email protected]> wrote: > Thanks for the pointer. Much nicer with the patch: > INFO : Operation: 'beancount.parser.parser' > Time: 1415 ms > INFO : Operation: 'beancount.ops.balance' > Time: 1054 ms > INFO : Operation: 'beancount.loader (total)' > Time: 33385 ms > > I'd like to eventually see if I can help get this down to under 10 sec, > which makes it more usable. I'm running on a Core i7. > > On Sunday, October 1, 2017 at 3:09:13 PM UTC-7, Martin Blais wrote: >> >> Can you try this patch? >> https://bitbucket.org/blais/beancount/issues/191/beancountco >> reinventory-should-be-a-dict >> >> >> >> On Sun, Oct 1, 2017 at 5:54 PM, <[email protected]> wrote: >> >>> Hi Martin, >>> So my beancount file takes a humongous time to load: about 2 minutes. >>> >>> bean-check -v shows: >>> INFO : Operation: 'beancount.parser.parser' >>> Time: 1412 ms >>> INFO : Operation: 'beancount.ops.balance' >>> Time: 9500 ms >>> INFO : Operation: 'beancount.loader (total)' >>> Time: 110842 ms >>> >>> My hacky solution this far has been to largely disable one of my source >>> files (I have one file per account), which contains about 1.5k transactions >>> out of a total of about 30k transactions. This cuts it down to about 18 >>> seconds. The 1.5k transactions belong to an investment account and thus >>> have stock purchases (and a few sells). If I remove all the balance >>> assertions in it, the beancount.loader (total) goes to about 95 seconds. >>> >>> I used a python profiler to further understand this: >>> https://ibb.co/iCTjow >>> >>> 61% of the time goes in 'add_amount()'. I'll take a deeper look later >>> on, but thought I'd post here and see if you had any feedback. >>> >>> Thanks! >>> >>> On Saturday, September 12, 2015 at 2:14:20 PM UTC-7, Martin Blais wrote: >>>> >>>> The load time varies depending on the size of the file. >>>> Mine takes about three seconds right now, as I maintain my entire >>>> history in a single file. >>>> If you want to see the breakdown of which stages take how long, run >>>> >>>> bean-check -v <filename> >>>> >>>> I have spent almost no time optimizing anything so far, but I'm >>>> confident I will eventually be able to cut the load time in half. I think a >>>> sensible goal would be to ensure 10 years worth of typical transactions >>>> parse in under 1 sec. >>>> >>>> >>>> On Sat, Sep 12, 2015 at 5:40 AM, <[email protected]> wrote: >>>> >>>>> I have build a script which basically sums up outstanding expenses for >>>>> a specific customer account. >>>>> >>>>> Each time I call this script, I feel it takes too long to execute. >>>>> time says 0.18 real 0.16 user 0.01 sys >>>>> >>>>> bean-query has similar performance, especially if I call it several >>>>> times in a script. >>>>> >>>>> Is this specific to my environment? Who can confirm a similar >>>>> experience? >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Beancount" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/beancount/4c19e464-3e87-41 >>>>> ec-941d-80ace9f9eb7a%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/beancount/4c19e464-3e87-41ec-941d-80ace9f9eb7a%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Beancount" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> To view this discussion on the web visit https://groups.google.com/d/ms >>> gid/beancount/3fd100db-40f4-4157-8a78-eb31f4248c05%40googlegroups.com >>> <https://groups.google.com/d/msgid/beancount/3fd100db-40f4-4157-8a78-eb31f4248c05%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "Beancount" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/beancount/7c3e801f-565e-4fe2-9d50-9054bb6b23f6%40googlegroups.com > <https://groups.google.com/d/msgid/beancount/7c3e801f-565e-4fe2-9d50-9054bb6b23f6%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAK21%2BhO%2B%2Bwn1UP3WbGwOjxt2%2Bde3tAoVm08a3m8hQKJ%3Dnzh_mQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
