[
https://issues.apache.org/jira/browse/IGNITE-22637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov updated IGNITE-22637:
--------------------------------------
Description:
To select a compaction time, each node should provide its MinimalRequiredTime
to compaction coordinator.
h4. Prerequisites:
In {{PartitionListener}} using partitionDataStorage.flush(false) store
{{minActiveTxBeginTime}} into another transient field (for example
{{snapshottedMinActiveTxBeginTime}}).
Need to investigate how to obtain this {{snapshottedMinActiveTxBeginTime}} from
{{CatalogCompactionRunner}} in production code (may be introduce necessary
methods). Example of this in test code can be found in
{{ItCatalogCompactionTest.ensureTimestampStoredInAllReplicas}}
h4. Compute minimum required time
1. Get current value of low watermark.
2. Get min of "snapshotted" minActiveTxTime across all partitions (see
{{PartitionListener.minActiveTxBeginTime}}.
Take min of 1., 2 and return it in
{{CatalogCompactionRunner.determineLocalMinimumRequiredTime}}
h4. Additional notes
Compaction must start only under the following conditions:
1. no active builds of indexes -- check for indexes are done on coordinator node
2. no active rebalance referring an old versions -- rebalance started with the
latest version (this point is moved to
https://issues.apache.org/jira/browse/IGNITE-23059)
Depending on the complexity, the last part may be implemented as a separate
ticket, but compaction should not be enabled until all parts are implemented.
was:
To select a compaction time, each node should provide its MinimalRequiredTime
to compaction coordinator.
h4. Prerequisites:
In {{PartitionListener}} using partitionDataStorage.flush(false) store
{{minActiveTxBeginTime}} into another transient field (for example
{{snapshottedMinActiveTxBeginTime}}).
Need to investigate how to obtain this {{snapshottedMinActiveTxBeginTime}} from
{{CatalogCompactionRunner}} in production code (may be introduce necessary
methods). Example of this in test code can be found in
{{ItCatalogCompactionTest.ensureTimestampStoredInAllReplicas}}
h4. Compute minimum required time
1. Get current value of low watermark.
2. Get min of "snapshotted" minActiveTxTime across all partitions (see
{{PartitionListener.minActiveTxBeginTime}}.
Take min of 1., 2 and return it in
{{CatalogCompactionRunner.determineLocalMinimumRequiredTime}}
h4. Additional notes
Compaction must start only under the following conditions:
1. no active builds of indexes -- check for indexes are done on coordinator node
2. no active rebalance referring an old versions -- rebalance started with the
latest version
Depending on the complexity, the last part may be implemented as a separate
ticket, but compaction should not be enabled until all parts are implemented.
> Catalog compaction. Implement remote node part of compaction flow
> -----------------------------------------------------------------
>
> Key: IGNITE-22637
> URL: https://issues.apache.org/jira/browse/IGNITE-22637
> Project: Ignite
> Issue Type: Improvement
> Reporter: Maksim Zhuravkov
> Assignee: Maksim Zhuravkov
> Priority: Major
> Labels: ignite-3
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> To select a compaction time, each node should provide its MinimalRequiredTime
> to compaction coordinator.
> h4. Prerequisites:
> In {{PartitionListener}} using partitionDataStorage.flush(false) store
> {{minActiveTxBeginTime}} into another transient field (for example
> {{snapshottedMinActiveTxBeginTime}}).
> Need to investigate how to obtain this {{snapshottedMinActiveTxBeginTime}}
> from {{CatalogCompactionRunner}} in production code (may be introduce
> necessary methods). Example of this in test code can be found in
> {{ItCatalogCompactionTest.ensureTimestampStoredInAllReplicas}}
> h4. Compute minimum required time
> 1. Get current value of low watermark.
> 2. Get min of "snapshotted" minActiveTxTime across all partitions (see
> {{PartitionListener.minActiveTxBeginTime}}.
> Take min of 1., 2 and return it in
> {{CatalogCompactionRunner.determineLocalMinimumRequiredTime}}
> h4. Additional notes
> Compaction must start only under the following conditions:
> 1. no active builds of indexes -- check for indexes are done on coordinator
> node
> 2. no active rebalance referring an old versions -- rebalance started with
> the latest version (this point is moved to
> https://issues.apache.org/jira/browse/IGNITE-23059)
> Depending on the complexity, the last part may be implemented as a separate
> ticket, but compaction should not be enabled until all parts are implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
