[
https://issues.apache.org/jira/browse/IGNITE-23293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Pligin reassigned IGNITE-23293:
----------------------------------------
Assignee: Kirill Tkalenko
> Implement the algorithm of Metastorage Compaction Trigger
> ---------------------------------------------------------
>
> Key: IGNITE-23293
> URL: https://issues.apache.org/jira/browse/IGNITE-23293
> Project: Ignite
> Issue Type: Improvement
> Reporter: Kirill Tkalenko
> Assignee: Kirill Tkalenko
> Priority: Major
> Labels: ignite-3
>
> We need to implement the algorithm of Metastorage Compaction Trigger:
> # Metastorage leader waits locally for the start of scheduled compaction.
> The interval is set in the configuration.
> # Metastorage leader locally calculates revision for compaction: it takes the
> current safe time and subtracts the data availability time (taken from the
> configuration) and uses that timestamp to get the revision.
> # If the revision is less than or equal to the last compacted revision, then
> go to point 6.
> # Metastorage leader creates and sends a raft command with a revision for
> compaction. Each node processes the raft command and does the following:
> ## Start processing the raft command.
> ## Locally saves the compaction revision to the metastorage KV storage meta
> (as well as the revision and update counter of the metastorage).
> ## Add a task (A) to the WatchEvent processing queue and complete the
> processing of the raft command (release the thread for processing raft
> commands).
> Explanation: Each WatchEvent starts to be processed after the previous one is
> completed, we will add our task that will start to be executed after the
> previous WatchEvent is processed, and the next WatchEvent processing will
> start after our task is completed. Thus, when the WatchEvent is being
> processed, we will not change the compaction revision in memory that will
> affect the CompactedException throw.
> ## In the task (A), set the compaction revision in memory, add a new task (B)
> to start local compaction.
> Explanation: From now on, CompactedException will be thrown when trying to
> read from the metastorage at this or lower revision. More details will be
> below after the algorithm.
> ## In the task (B), locally collect all cursors, getters and publishers for
> revisions less than or equal to the new compaction revision.
> ## After their completion, the local compaction starts (C).
> ## Notify the CompactionListeners that the compaction has been completed for
> the new compaction revision.
> # Metastorage leader locally gets notification of the completion of the local
> compaction for the new revision.
> # Metastorage leader locally schedules a new start of compaction.
> The interval is taken from the configuration.
> Notes:
> * During WatchEvent processing, the compaction revision will not change in
> memory in a way that could cause an CompactedException to be thrown during
> event processing.
> * For now, we can make the configuration, for example, a system property or
> something like that, later in IGNITE-23279.
> * For now, we can't to turn on trigger in production code, we'll do it in
> IGNITE-23280.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
