[
https://issues.apache.org/jira/browse/IGNITE-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirill Tkalenko updated IGNITE-16657:
-------------------------------------
Description:
At the moment, after each checkpoint, delta files are created for each modified
partition (which have dirty pages).
We need to implement merging the delta file into the main partition file to
reduce read amplification.
Suggested solution:
* Merge to make multi-thread, the number of threads to set in the configuration;
* Each thread processes one delta file at a time;
* Merge to start as soon as there are delta files.
The question remains, what to do when a checkpoint and merge of delta files are
working at the same time, who should be given priority and how to deal with it?
It is proposed to think over and do in IGNITE-17446.
was:
At the moment, after each checkpoint, delta files are created for each modified
partition (which have dirty pages).
We need to implement merging the delta file into the main partition file to
reduce read amplification.
Suggested solution:
* Merge to make multi-thread, the number of threads to set in the configuration;
* Each thread processes one delta file at a time;
* Merge to start as soon as there are delta files.
The question remains, what to do when a checkpoint and merge of delta files are
working at the same time, who should be given priority and how to deal with it?
It is proposed to think over and do in 123.
> [Native Persistence 3.0] Implement partitions chunks merger
> -----------------------------------------------------------
>
> Key: IGNITE-16657
> URL: https://issues.apache.org/jira/browse/IGNITE-16657
> Project: Ignite
> Issue Type: Improvement
> Reporter: Ivan Bessonov
> Assignee: Kirill Tkalenko
> Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
>
> At the moment, after each checkpoint, delta files are created for each
> modified partition (which have dirty pages).
> We need to implement merging the delta file into the main partition file to
> reduce read amplification.
> Suggested solution:
> * Merge to make multi-thread, the number of threads to set in the
> configuration;
> * Each thread processes one delta file at a time;
> * Merge to start as soon as there are delta files.
> The question remains, what to do when a checkpoint and merge of delta files
> are working at the same time, who should be given priority and how to deal
> with it? It is proposed to think over and do in IGNITE-17446.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
