Sounds good. Here is the new profile after it came down from 110s to 33s: http://ibb.co/j62cQb
I'll take a closer look later. On Sunday, October 1, 2017 at 4:11:10 PM UTC-7, Martin Blais wrote: > > Let's profile further, and if needed, to write the parts most called in C. > > > On Sun, Oct 1, 2017 at 6:22 PM, <[email protected] <javascript:>> wrote: > >> Thanks for the pointer. Much nicer with the patch: >> INFO : Operation: 'beancount.parser.parser' >> Time: 1415 ms >> INFO : Operation: 'beancount.ops.balance' >> Time: 1054 ms >> INFO : Operation: 'beancount.loader (total)' >> Time: 33385 ms >> >> I'd like to eventually see if I can help get this down to under 10 sec, >> which makes it more usable. I'm running on a Core i7. >> >> On Sunday, October 1, 2017 at 3:09:13 PM UTC-7, Martin Blais wrote: >>> >>> Can you try this patch? >>> >>> https://bitbucket.org/blais/beancount/issues/191/beancountcoreinventory-should-be-a-dict >>> >>> >>> >>> On Sun, Oct 1, 2017 at 5:54 PM, <[email protected]> wrote: >>> >>>> Hi Martin, >>>> So my beancount file takes a humongous time to load: about 2 minutes. >>>> >>>> bean-check -v shows: >>>> INFO : Operation: 'beancount.parser.parser' >>>> Time: 1412 ms >>>> INFO : Operation: 'beancount.ops.balance' >>>> Time: 9500 ms >>>> INFO : Operation: 'beancount.loader (total)' >>>> Time: 110842 ms >>>> >>>> My hacky solution this far has been to largely disable one of my source >>>> files (I have one file per account), which contains about 1.5k >>>> transactions >>>> out of a total of about 30k transactions. This cuts it down to about 18 >>>> seconds. The 1.5k transactions belong to an investment account and thus >>>> have stock purchases (and a few sells). If I remove all the balance >>>> assertions in it, the beancount.loader (total) goes to about 95 seconds. >>>> >>>> I used a python profiler to further understand this: >>>> https://ibb.co/iCTjow >>>> >>>> 61% of the time goes in 'add_amount()'. I'll take a deeper look later >>>> on, but thought I'd post here and see if you had any feedback. >>>> >>>> Thanks! >>>> >>>> On Saturday, September 12, 2015 at 2:14:20 PM UTC-7, Martin Blais wrote: >>>>> >>>>> The load time varies depending on the size of the file. >>>>> Mine takes about three seconds right now, as I maintain my entire >>>>> history in a single file. >>>>> If you want to see the breakdown of which stages take how long, run >>>>> >>>>> bean-check -v <filename> >>>>> >>>>> I have spent almost no time optimizing anything so far, but I'm >>>>> confident I will eventually be able to cut the load time in half. I think >>>>> a >>>>> sensible goal would be to ensure 10 years worth of typical transactions >>>>> parse in under 1 sec. >>>>> >>>>> >>>>> On Sat, Sep 12, 2015 at 5:40 AM, <[email protected]> wrote: >>>>> >>>>>> I have build a script which basically sums up outstanding expenses >>>>>> for a specific customer account. >>>>>> >>>>>> Each time I call this script, I feel it takes too long to execute. >>>>>> time says 0.18 real 0.16 user 0.01 sys >>>>>> >>>>>> bean-query has similar performance, especially if I call it several >>>>>> times in a script. >>>>>> >>>>>> Is this specific to my environment? Who can confirm a similar >>>>>> experience? >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Beancount" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To post to this group, send email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/beancount/4c19e464-3e87-41ec-941d-80ace9f9eb7a%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/beancount/4c19e464-3e87-41ec-941d-80ace9f9eb7a%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Beancount" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/beancount/3fd100db-40f4-4157-8a78-eb31f4248c05%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/beancount/3fd100db-40f4-4157-8a78-eb31f4248c05%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Beancount" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/beancount/7c3e801f-565e-4fe2-9d50-9054bb6b23f6%40googlegroups.com >> >> <https://groups.google.com/d/msgid/beancount/7c3e801f-565e-4fe2-9d50-9054bb6b23f6%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Beancount" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/beancount/7d9809a0-65b2-424d-a4b6-619c46343303%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
