[
https://issues.apache.org/jira/browse/IGNITE-12617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Vinogradov updated IGNITE-12617:
--------------------------------------
Description:
Since IGNITE-9913, new-topology operations allowed immediately after
cluster-wide recovery finished.
But is there any reason to wait for a cluster-wide recovery if only one node
failed?
In this case, we should recover only the failed node's backups.
Unfortunately, {{RendezvousAffinityFunction}} tends to spread the node's backup
partitions to the whole cluster. In this case, we, obviously have to perform
cluster-wide recovery on switch.
But what if only some nodes will be the backups for every primary?
In case nodes combined into virtual cells where, for each partition, backups
located at the same cell with primaries, it's possible to finish the switch
outside the affected cell before tx recovery finish.
This optimization will allow us to start and even finish new operations outside
the failed cell without cluster-wide switch finish waiting.
In other words, switch (when left/fail + baseline + rebalanced) will have
little effect on the operation's (not related to failed cell) latency.
Assumptions
- We should wait for tx recovery before finishing the global switch.
- We should wait for replicated caches recovery globally before finishing
switch locally.
- Upcoming replicated caches operations and operations related to the broken
cell will require a cluster-wide switch finish to be committed.
was:
Since IGNITE-9913, new-topology operations allowed immediately after
cluster-wide recovery finished.
But is there any reason to wait for a cluster-wide recovery if only one node
failed?
In this case, we should recover only the failed node's backups.
Unfortunately, RendezvousAffinityFunction tends to spread some node's backup
partitions to the whole cluster. In this case, we, obviously have to perform
cluster-wide recovery on switch.
But what if only some nodes will be the backups for every primary?
In case nodes combined into virtual cells where, for each partition, backups
located at the same cell with primaries, it's possible to finish the switch
outside the affected cell before tx recovery finish.
This optimization will allow us to start and even finish new operations outside
the failed cell immediately.
In other words
- We should wait for tx recovery before finishing the global switch.
- We should wait for replicated caches recovery globally.
- As to partitioned caches, we have to minimize the waiting group to allow
upcoming operations where possible during the switch.
> PME-free switch should wait for recovery only at affected nodes.
> ----------------------------------------------------------------
>
> Key: IGNITE-12617
> URL: https://issues.apache.org/jira/browse/IGNITE-12617
> Project: Ignite
> Issue Type: Task
> Reporter: Anton Vinogradov
> Assignee: Anton Vinogradov
> Priority: Major
> Labels: iep-45
> Fix For: 2.9
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Since IGNITE-9913, new-topology operations allowed immediately after
> cluster-wide recovery finished.
> But is there any reason to wait for a cluster-wide recovery if only one node
> failed?
> In this case, we should recover only the failed node's backups.
> Unfortunately, {{RendezvousAffinityFunction}} tends to spread the node's
> backup partitions to the whole cluster. In this case, we, obviously have to
> perform cluster-wide recovery on switch.
> But what if only some nodes will be the backups for every primary?
> In case nodes combined into virtual cells where, for each partition, backups
> located at the same cell with primaries, it's possible to finish the switch
> outside the affected cell before tx recovery finish.
> This optimization will allow us to start and even finish new operations
> outside the failed cell without cluster-wide switch finish waiting.
> In other words, switch (when left/fail + baseline + rebalanced) will have
> little effect on the operation's (not related to failed cell) latency.
> Assumptions
> - We should wait for tx recovery before finishing the global switch.
> - We should wait for replicated caches recovery globally before finishing
> switch locally.
> - Upcoming replicated caches operations and operations related to the broken
> cell will require a cluster-wide switch finish to be committed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
