This copy is costly but might be avoidable, but it will perhaps be sensitive to remove it: https://bitbucket.org/blais/beancount/src/6b61612e7f44580a9017c3c05d3efdff2981be81/beancount/parser/booking_full.py?at=default&fileviewer=file-view-default#booking_full.py-553
On Sun, Oct 1, 2017 at 11:15 PM, <[email protected]> wrote: > Sounds good. Here is the new profile after it came down from 110s to 33s: > http://ibb.co/j62cQb > > I'll take a closer look later. > > On Sunday, October 1, 2017 at 4:11:10 PM UTC-7, Martin Blais wrote: >> >> Let's profile further, and if needed, to write the parts most called in C. >> >> >> On Sun, Oct 1, 2017 at 6:22 PM, <[email protected]> wrote: >> >>> Thanks for the pointer. Much nicer with the patch: >>> INFO : Operation: 'beancount.parser.parser' >>> Time: 1415 ms >>> INFO : Operation: 'beancount.ops.balance' >>> Time: 1054 ms >>> INFO : Operation: 'beancount.loader (total)' >>> Time: 33385 ms >>> >>> I'd like to eventually see if I can help get this down to under 10 sec, >>> which makes it more usable. I'm running on a Core i7. >>> >>> On Sunday, October 1, 2017 at 3:09:13 PM UTC-7, Martin Blais wrote: >>>> >>>> Can you try this patch? >>>> https://bitbucket.org/blais/beancount/issues/191/beancountco >>>> reinventory-should-be-a-dict >>>> >>>> >>>> >>>> On Sun, Oct 1, 2017 at 5:54 PM, <[email protected]> wrote: >>>> >>>>> Hi Martin, >>>>> So my beancount file takes a humongous time to load: about 2 minutes. >>>>> >>>>> bean-check -v shows: >>>>> INFO : Operation: 'beancount.parser.parser' >>>>> Time: 1412 ms >>>>> INFO : Operation: 'beancount.ops.balance' >>>>> Time: 9500 ms >>>>> INFO : Operation: 'beancount.loader (total)' >>>>> Time: 110842 ms >>>>> >>>>> My hacky solution this far has been to largely disable one of my >>>>> source files (I have one file per account), which contains about 1.5k >>>>> transactions out of a total of about 30k transactions. This cuts it down >>>>> to >>>>> about 18 seconds. The 1.5k transactions belong to an investment account >>>>> and >>>>> thus have stock purchases (and a few sells). If I remove all the balance >>>>> assertions in it, the beancount.loader (total) goes to about 95 seconds. >>>>> >>>>> I used a python profiler to further understand this: >>>>> https://ibb.co/iCTjow >>>>> >>>>> 61% of the time goes in 'add_amount()'. I'll take a deeper look later >>>>> on, but thought I'd post here and see if you had any feedback. >>>>> >>>>> Thanks! >>>>> >>>>> On Saturday, September 12, 2015 at 2:14:20 PM UTC-7, Martin Blais >>>>> wrote: >>>>>> >>>>>> The load time varies depending on the size of the file. >>>>>> Mine takes about three seconds right now, as I maintain my entire >>>>>> history in a single file. >>>>>> If you want to see the breakdown of which stages take how long, run >>>>>> >>>>>> bean-check -v <filename> >>>>>> >>>>>> I have spent almost no time optimizing anything so far, but I'm >>>>>> confident I will eventually be able to cut the load time in half. I >>>>>> think a >>>>>> sensible goal would be to ensure 10 years worth of typical transactions >>>>>> parse in under 1 sec. >>>>>> >>>>>> >>>>>> On Sat, Sep 12, 2015 at 5:40 AM, <[email protected]> wrote: >>>>>> >>>>>>> I have build a script which basically sums up outstanding expenses >>>>>>> for a specific customer account. >>>>>>> >>>>>>> Each time I call this script, I feel it takes too long to execute. >>>>>>> time says 0.18 real 0.16 user 0.01 sys >>>>>>> >>>>>>> bean-query has similar performance, especially if I call it several >>>>>>> times in a script. >>>>>>> >>>>>>> Is this specific to my environment? Who can confirm a similar >>>>>>> experience? >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Beancount" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To post to this group, send email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/beancount/4c19e464-3e87-41 >>>>>>> ec-941d-80ace9f9eb7a%40googlegroups.com >>>>>>> <https://groups.google.com/d/msgid/beancount/4c19e464-3e87-41ec-941d-80ace9f9eb7a%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Beancount" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/beancount/3fd100db-40f4-41 >>>>> 57-8a78-eb31f4248c05%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/beancount/3fd100db-40f4-4157-8a78-eb31f4248c05%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Beancount" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> To view this discussion on the web visit https://groups.google.com/d/ms >>> gid/beancount/7c3e801f-565e-4fe2-9d50-9054bb6b23f6%40googlegroups.com >>> <https://groups.google.com/d/msgid/beancount/7c3e801f-565e-4fe2-9d50-9054bb6b23f6%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "Beancount" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/beancount/7d9809a0-65b2-424d-a4b6-619c46343303%40googlegroups.com > <https://groups.google.com/d/msgid/beancount/7d9809a0-65b2-424d-a4b6-619c46343303%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/CAK21%2BhNfQ5AOJz9b3kpjiFfzbY%2BoADm6LHsyd740NyU9cpnfbA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
