On Sunday, 12 June 2016 22:36:41 UTC-7, Pavel wrote: > > Greetings, > > I am evaluating ledger for a use case where a large accounting dataset > contains tens of thousands transactions every month, over several years. > Transactions are mostly currency conversions, and my initial search > specifically led me to ledger due to its flexible commodity handling. A few > questions related to the large size of the journal: > > (1) Would it be practical to try the program with such a large transaction > dataset at all? Anyone tried over 10^5 - 10^6 transactions? >
Out of curiosity, I wrote a small script in Awk to generate 10^6 transactions for testing (attached). The transactions it generates just move random amounts in random currencies amongst accounts, so it's an oversimplification of what you want to do. I imagine that adding conversions will increase processing time linearly. On my one-step-above-a-netbook ( Celeron N2840 @ 2.1 GHz × 2 ), running a simple balance report took about 40s, while subtotalling increased time as the number of calculations increased: $ time ledger -f ledger.dat bal > /dev/null real 0m39.225s user 0m36.176s sys 0m1.404s $ time ledger -f ledger.dat --weekly bal > /dev/null real 0m50.521s user 0m47.956s sys 0m1.380s $ time ledger -f ledger.dat --daily bal > /dev/null real 1m0.351s user 0m54.828s sys 0m1.608s > (2) I'd have to script json (current data) parsing to ledger journal > format. Any recommended alternatives to input formats (e.g. xml mentioned > in some places) to speed up transaction parsing? Or would it be as fast > with the human-readable journal input text format? > I have no idea, but I would think a database backend would be faster at this scale. > (3) If updating the journal, say, monthly, is there any way to cache > previous month's calculations or would the entire updated ledger have to be > parsed and recalculated each time? > Yes. Run an equity report ( http://ledger-cli.org/3.0/doc/ledger3.html#Archiving-Previous-Years) to roll up the books prior to a set time and create an "opening balances" entry to start the new month: $ time ledger -f ledger.dat --end 2016/08/01 equity > /dev/null real 0m50.588s user 0m45.696s sys 0m1.580s > > Any recommendations would be helpful as I do like the cleanness of the > program and hope to make it suitable for this use case. Thanks. > > -- --- You received this message because you are subscribed to the Google Groups "Ledger" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
benchmark.org
Description: Binary data
