Kohei Yoshida пишет:
Fixing i55266 (and probably i97886) needs a non-trivial refactoring
since the reason for such large memory footprint may be a design issue.
Current data pilot's result computation code uses a recursive algorithm,
and each recursion instantiates a new set of objects. Unfortunately,
when the number of unique field member values becomes large, which is
certainly the case for the above referenced issues, the size of
instantiated objects grows, and the memory usage spikes. Note that this
algorithm is *not* weak against the total size of a source data, but is
weak against the large number of unique field member values.
We could first try to minimize memory use without changing the algorithm
itself (by reusing some of the same tricks I used to reduce memory usage
in the cache table). But if that turns out to be insufficient, then we
may have to re-write the algorithm itself, in which case the required
effort becomes pretty significant.
Anyway, this is my initial thought on this issue. Looking deeper may
reveal more detailed view of the problem.
Kohei,
thank you very much for our analisys and explanation!
Now I feel a whole lot better knowing why we crash
and why issue is not fixed for this long.
With best regards,
Kirill Palagin.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
