https://bugs.kde.org/show_bug.cgi?id=524780
Bug ID: 524780
Summary: Income/Expense report with weekly columns takes ~50 s
to open in 5.2.2 (2-3 s in 5.1.3)
Classification: Applications
Product: kmymoney
Version First 5.2.2
Reported In:
Platform: Ubuntu
OS: Linux
Status: REPORTED
Severity: normal
Priority: NOR
Component: reports
Assignee: [email protected]
Reporter: [email protected]
Target Milestone: ---
SUMMARY
One of my reports has become extremely slow after updating to 5.2.2.
It is an “Income and Expenses” report over my whole data range (about 17
years), with the columns set to weeks. In 5.2.2 it takes about 50 seconds until
it shows up. The very same report in 5.1.3 opens in 2-3 seconds. Exporting that
report to HTML takes just as long, about 50 seconds.
Other reports are fine. The same report with monthly columns opens in 1 second,
and the weekly report opens in 1 second as well if I limit it to the last year.
So it only gets slow when the report has many weekly columns.
The report itself is not big: the exported HTML has 31 rows and 871 columns.
STEPS TO REPRODUCE
Create an Income/Expense report over a long date range (here: ~17 years).
Set the column type to “Weeks”.
Open the report (or export it as HTML).
OBSERVED RESULT
~50 seconds until the report appears. One CPU core at 100 % the whole time.
EXPECTED RESULT
Comparable to 5.1.3 (2-3 s), or at least comparable to the monthly report on
the same data (1 s).
SOFTWARE/OS VERSIONS
KMyMoney 5.2.2 (Flatpak org.kde.kmymoney, Qt 6.10.3, Alkimia 8.2.1) - slow
KMyMoney 5.1.3 (Ubuntu 24.04 package, Qt 5.15, Alkimia 8.1.2) - fast
Linux 6.8.0, Ubuntu 24.04
NOTE ON THE ANALYSIS BELOW
I tried to narrow this down myself with the help of Claude (an AI assistant),
which read the source, ran measurements and profiled the running Flatpak build.
It did not find the actual cause. Several plausible-sounding explanations were
produced and then disproved one after another by measurement - including its
own first and most confident hypothesis, that the switch from WebEngine to
QTextBrowser was to blame.
So please treat the section “SUGGESTED DIRECTION” as guesswork. What I do think
is worth your time is the measured data and the list of things that were ruled
out, so you do not have to walk down those four dead ends yourselves. If any of
it looks wrong, trust your own judgement over it - I cannot verify the
reasoning myself.
MEASUREMENTS
Timing depends on the internal column count, not on the number of transactions
Configuration internal columns transactions time
Months, full range ~200 all 1 s
Weeks, 1 year 365 (day grid) 1 year 1 s
Days, 1 year 365 1 year 1-2 s
Weeks, full range ~6100 all 50 s
Note that “Weeks, 1 year” and “Months, full range” have very different
transaction counts but the same runtime, while “Weeks, full range” differs from
“Months, full range” only in the column count. The cost is essentially linear
in the internal column count, at roughly 5-8 ms per column (with only 31
account rows, i.e. on the order of 250 us per grid cell).
Weekly reports build the grid at daily resolution
In PivotTable::init(), m_numColumns is initially the number of days when
m_config.isColumnsAreDays() is true (columnValue() returns day differences);
collapseColumns() folds 7 day columns into one week column afterwards
(kmymoney/plugins/views/reports/core/pivottable.cpp, collapseColumns()).
A weekly report over 17 years therefore works on ~6100 columns instead of ~871
- seven times more than the result needs. Monthly reports do not pay this,
because columnValue() computes in months directly. This explains why weekly
(and daily) reports are the ones affected.
Profile: the time is spent in money value operations, not in rendering
Poor man’s profiler (gdb, 2254 samples over 180 s, covering both a report open
and an HTML export) against the running 5.2.2 Flatpak:
=== innermost frame (function) ===
716 31.8% libalkimia6: AlkValue::AlkValue(AlkValue const&)
705 31.3% libc: sem_trywait (idle)
613 27.2% libalkimia6: AlkValue::~AlkValue()
30 1.3% libQt6Gui (all frames combined)
=== library anywhere in the stack ===
72.3% reportsview.so
60.6% libalkimia6
46.8% libkmm_mymoney
Excluding idle samples, ~86 % of the CPU time is spent constructing and
destroying AlkValue objects, called from the reports plugin. Qt’s text layout
accounts for 1.3 %.
Caveat: the Flatpak plugin is stripped, so symbols inside reportsview.so could
not be resolved; the exact loop could not be identified from the outside.
Symbol attribution inside libalkimia6 is to the nearest exported symbol and may
be off by one function.
WHAT WAS RULED OUT (each by measurement, not by reading)
The QTextBrowser renderer (5.1.3 used WebEngine, 5.2 uses KMMTextBrowser).
A standalone Qt harness rendering report-shaped tables shows QTextDocument
layout to be linear in cell count at ~30 us/cell (with the KMyMoney CSS
applied): 208000 cells = 6.2 s, 24392 cells (the actual report size) well under
1 s. Additionally, the HTML export - which never touches the renderer - is just
as slow, and libQt6Gui is at 1.3 % in the profile.
Missing fixed column widths in the generated HTML. Measured: no effect
whatsoever (auto layout vs. explicit widths, identical timings).
collapseColumns() / the day grid folding itself. A standalone reproduction
of the nested QMap/QList grid with 31 rows and 6100 day columns folds in 6 ms
(15 ms with an extra shared copy held). Not the bottleneck.
Implicit-sharing detach cascade in the chained
m_grid[og][ig][row][eActual][column] += access in assignCell(). A standalone
reproduction with 20000 splits at 6100 columns runs in 9-20 ms - under both Qt
5.15 and Qt 6.4. Not the bottleneck.
Alkimia. AlkValue’s copy constructor is unchanged between 8.1.2 and 8.2.1
(d(val.d), QSharedDataPointer); the diff of alkvalue.cpp between the two tags
contains only the QRegExp → QRegularExpression port and unrelated fixes.
SUGGESTED DIRECTION FOR DEVELOPERS
Two independent things are worth looking at:
Why is a single grid column so expensive? ~5-8 ms per column with 31 rows
is far more than the folding and the currency conversion should cost. With
symbols available, a profile of PivotTable::init() should show this
immediately. The loops that run at day resolution are the transaction loop and
collapseColumns(); the loops that run after folding are
convertToDeepCurrency(), convertToBaseCurrency() (one price lookup per cell)
and calculateTotals().
Does the day grid need to exist at all? For weekly/daily pitches the grid
is allocated and processed at daily resolution and only then folded. Assigning
values directly to the target column would remove a factor of 7 for weekly
reports independently of point 1.
UNRELATED FINDINGS NOTICED WHILE READING
ReportAccount::operator=
(kmymoney/plugins/views/reports/core/reportaccount.cpp) copies its own members
but never calls MyMoneyAccount::operator=, so the base class part (id, name,
type, currency) is not copied on assignment.
PivotTable::renderHTML() deep-copies the entire grid into a
QList<PivotOuterGroup> just to sort the outer groups; the code comments on this
itself. Sorting pointers or iterators would avoid it. (Not performance-relevant
at the measured sizes, but it is unnecessary work.)
--
You are receiving this mail because:
You are watching all bug changes.
