[
https://issues.apache.org/jira/browse/IGNITE-23413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17895644#comment-17895644
]
Ivan Bessonov commented on IGNITE-23413:
----------------------------------------
What I think needs to be done:
* We should fix current local meta-storage version and use it for further
reads.
* We should read all pending and planned assignments, there's a timestamp
stored in each of them. We calculate the minimal one.
* If there are no pending and planned assignments, we might return the
timestamp that's associated with meta-storage revision.
This approach almost works, there are situations with nuances. These are the
cases where we're in-between some operation. Let's examine them:
* Zone is created/altered, but assignments are not yet saved. This is the case
when assignments timestamp is below latest zone's timestamp.
** For ALTER, returning an older timestamp from assignments is not a problem,
it'll eventually become more recent.
** For CREATE, we should probably determine if assignments are not yet saved,
and use the timestamp from catalog.
* A list of data nodes is updated, but assignments are not yet re-calculated
because of timeout.
** Current code uses "latest" catalog state when it transforms data nodes into
assignments, so it is safe to use timestamp of latest catalog version.
There's a Jira that aims to fix it:
https://issues.apache.org/jira/browse/IGNITE-22723. This means that current
approach might not work in the future.
Anyway, considering everything from the above, there's one situation that we
must keep in mind:
* DZ is updated at catalog version 15.
* Assignments are calculated for the same exact catalog version.
* Nothing changes for a long time. Several days, for example.
* During that time, catalog version increases and becomes 75, for example.
If nothing changes, we should be able to remove versions 15-74, because DZ
settings from versions 15 and 75 are identical.
It seems like the proposed algorithm works exactly like we need.
> Catalog compaction. Component to determine minimum catalog version required
> by rebalance.
> -----------------------------------------------------------------------------------------
>
> Key: IGNITE-23413
> URL: https://issues.apache.org/jira/browse/IGNITE-23413
> Project: Ignite
> Issue Type: Improvement
> Reporter: Pavel Pereslegin
> Assignee: Ivan Bessonov
> Priority: Major
> Labels: ignite-3
>
> Each rebalance procedure uses specific catalog version, it "holds" the
> timestamp corresponding to the latest (at the moment of rebalancing start)
> version of the catalog
> To be able safely perform catalog compaction, we need to design and implement
> a component that can determine the minimum version required for active
> rebalances (to avoid deleting this version during compaction).
> {code:java}
> interface RebalanceMinimumRequiredTimeProvider {
> /**
> * Returns the minimum time required for rebalance,
> * or current timestamp if there are no active
> * rebalances and there is a guarantee that all rebalances
> * launched in the future will use catalog version
> * corresponding to the current time or greater.
> */
> long minimumRequiredTime();
> }
> {code}
> The component can be either global or local (whichever is easier to
> implement). This means that the compaction procedure can call the component
> on all nodes in the cluster and calculate the minimum.
> The component must be able to track rebalances that may be triggered during
> "replay" of the metastorage raft log.
> The component should return only monotonically increasing values.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
