[
https://issues.apache.org/jira/browse/IGNITE-18031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Bessonov resolved IGNITE-18031.
------------------------------------
Resolution: Invalid
> Implement background GC process for MV partition storages
> ---------------------------------------------------------
>
> Key: IGNITE-18031
> URL: https://issues.apache.org/jira/browse/IGNITE-18031
> Project: Ignite
> Issue Type: Improvement
> Reporter: Ivan Bessonov
> Priority: Major
> Labels: ignite-3
>
> Please refer to Epic for more details. Here I only describe thoughts about
> background GC process.
> h3. General thoughts
> Basic algorithm is the following:
>
> {code:java}
> RowId rowId = null;
> while (!partitionStopped) {
> if (rowId == null) rowId = minRowId(partId);
> rowId = partition.closestRowId(rowId);
> partition.gc(rowId);
> rowId = rodId.increment();
> }{code}
> Here I ignore a lot of technicalities and only show the main loop. We _could_
> implement it literally the same way, but that would be a very bad decision.
> Why:
> * it fully utilizes a single thread. That would mean a thread per each
> partition, which is unacceptable
> * it constantly reads the entire partition over and over again. I don't like
> that, it's a waste of resources, we'd rather prioritize reading data that
> user needs. There should be pauses between full runs, at least
> To address all of these issues, the job should be split into small batches
> (like we do in many other places) and every new batch should be put into a
> pool only when the current one is completed. This allows multiple partitions
> to utilize the same pool without a possibility of starvation.
> The part with the _pause_ should probably involve a scheduled pool. That
> seems enough for the first implementation.
> Other ideas and approaches will be discussed in other issues.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
