Ivan Bessonov created IGNITE-18033:
--------------------------------------
Summary: Implement cooperative GC of MV data during RAFT commands
execution
Key: IGNITE-18033
URL: https://issues.apache.org/jira/browse/IGNITE-18033
Project: Ignite
Issue Type: Improvement
Reporter: Ivan Bessonov
Please refer to the Epic for the basic description. Please also refer to
IGNITE-18031 for naive implementation details and general thoughts.
Technically, there is a possibility that the background GC process wouldn't
catch up if there's too much data being loaded into the system. Scanning
through the entire partition takes time, and only a small subset of data could
be under a constant stream of modification.
To account for that, each update can be preceded with the manual GC of that
row. In this case, there's less work for the background processor, and there's
an empirical sense that frequently updated data will be just as frequently
vacuumed, thus not allowing too much garbage to appear in the first place.
Now that's not a guarantee that there would be no problems at all, so we should
also think about other ways of cooperative GC.
Of course, it would be nice to have a queue of "old" rows ready for you to
clean. How feasible is it? It would have to be supported by every engine.
That's extra data and extra time for management. Looks pretty much exactly like
a TTL manager in Ignite 2.x. I assume that we don't implement anything like
that right now, starting with more naive approaches.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
