[
https://issues.apache.org/jira/browse/IGNITE-29088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18121485#comment-18121485
]
Ignite TC Bot commented on IGNITE-29088:
----------------------------------------
{panel:title=Branch: [pull/13627/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/13627/head] Base: [master] : New Tests
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#00008b}Cache 17{color} [[tests
1|https://ci2.ignite.apache.org/viewLog.html?buildId=9375079]]
* {color:#013220}IgniteCacheTestSuite15:
CachePartitionLossIgnorePolicyMapTest.testStaleMapBeforeLostPartitionOwned -
PASSED{color}
{panel}
[TeamCity *--> Run :: All*
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=9375090&buildTypeId=IgniteTests24Java8_RunAll]
{color:#ffffff}tcbot-analysis-comment chainBuildId=9375090
rerunBuildIds=none{color}
> Resend partition maps after detecting lost partitions under the IGNORE policy
> -----------------------------------------------------------------------------
>
> Key: IGNITE-29088
> URL: https://issues.apache.org/jira/browse/IGNITE-29088
> Project: Ignite
> Issue Type: Bug
> Reporter: Maksim Davydov
> Assignee: Maksim Davydov
> Priority: Major
> Labels: ise
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *Problem*
> In an in-memory cluster with the IGNORE partition loss policy (baseline
> auto-adjust on, timeout 0), a node leaves and takes the only copy of some
> partitions with it (backups = 0). Each lost partition is recreated empty on
> its new primary and owned there. Sometimes the coordinator never learns that:
> its partition map keeps the partition MOVING until the next exchange.
> *Impact*
> Until the next topology change, which in a stable cluster can be hours away,
> the coordinator and every node that takes its full map see no owner of the
> partition:
> - SQL queries from those nodes fail after the retry timeout:
> "Failed to map SQL query to topology during timeout: 30000ms". With a
> MOVING partition in the map, ReducePartitionMapper maps partitions by their
> owners and finds none. On the new primary, whose own map is right, the same
> query succeeds.
> - awaitPartitionMapExchange() in tests times out. Key operations and scan
> queries still work.
> This is not what IGNORE is meant to do: the policy resets a lost partition
> silently, no
> partition is LOST, and there is nothing to reset by hand.
> *Cause*
> With the exchange merge protocol, the new primary creates the lost partition
> as MOVING when it gets the coordinator's full message, and owns it a moment
> later in detectLostPartitions. The coordinator, in its own
> detectLostPartitions, already marks the partition OWNING for that node.
> If the new primary sends its partition map between these two steps, the map
> carries MOVING with a newer update sequence, and the coordinator takes it. A
> typical sender is the scheduled resend (scheduleResendPartitions(), 1.5 s
> after an eviction or a map change). Nothing sends the node's map again after
> it owns the partition: the result of detectLostPartitions is dropped in
> GridDhtPartitionsExchangeFuture#detectLostPartitions.
> Also, inside GridDhtPartitionTopologyImpl#detectLostPartitions the result
> reflects only the last lost partition.
> *How it shows*
> IgniteTopologyValidatorGridSplitCacheTest (32 nodes, 50 caches without
> backups) times out in awaitPartitionMapExchange() after stopping its
> configless node in 9 of 15 local runs. Small clusters rarely send a map
> inside that window, so IgniteCachePartitionLossPolicySelfTest (3 nodes)
> doesn't hit it.
> *Fix*
> When detectLostPartitions changes a local partition, a non-coordinator node
> sends its single map again. A new test,
> CachePartitionLossIgnorePolicyMapTest, makes the new primary send its map
> inside the window every time and checks that the coordinator's map matches
> the nodes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
