maksaska opened a new pull request, #13627:
URL: https://github.com/apache/ignite/pull/13627
### What goes wrong
Under the IGNORE loss policy, a node that leaves with the only copy of some
partitions leaves
them to be recreated empty on their new primary. With the exchange merge
protocol the new
primary creates such a partition as MOVING when it gets the coordinator's
full message and owns
it a moment later in `detectLostPartitions`. If the node sends its partition
map in between,
the coordinator takes MOVING over what it computed itself, and nothing sends
the map again:
```
new primary N coordinator
------------- -----------
full message -> p created as MOVING
scheduled resend fires -> single map, p=MOVING ->
detectLostPartitions: N
owns p (OWNING)
<- single map arrives,
newer updateSeq:
p=MOVING wins
detectLostPartitions -> p.own() = OWNING
(nothing is sent)
```
Until the next exchange the coordinator, and every node that takes its full
map, sees no owner
of `p`. SQL queries from those nodes fail after the retry timeout
(`Failed to map SQL query to topology during timeout: 30000ms`), and
`awaitPartitionMapExchange()` in tests times out. No partition is LOST, so
there is nothing
to reset.
### The change
- `GridDhtPartitionsExchangeFuture#detectLostPartitions` collects the cache
groups whose
topology reports a changed local partition, and a non-coordinator node
calls
`refreshPartitions` for them. The resent map has a newer update sequence
than any map built
before `own()`, so it wins. The coordinator doesn't resend: its own states
are already in its
map, and every node sets the same states for the other nodes when it
detects the loss.
- `GridDhtPartitionTopologyImpl#detectLostPartitions` no longer overwrites
`changed` in its
loop. The result now tells whether any local partition changed, not only
the last one.
- The resend also fires under the SAFE policies (a partition marked LOST)
and on activation
with lost partitions: at most one extra single map per affected node per
such exchange.
### Tests
- New `CachePartitionLossIgnorePolicyMapTest` (in `IgniteCacheTestSuite15`).
The new primary
refreshes its map right before it owns a lost partition, and that map is
held until the
coordinator finishes the exchange. The forced refresh stands in for a
scheduled resend that
fires inside the window in a real cluster. It fails 10 of 10 runs without
the fix
(`local=OWNING crd=MOVING`) and passes 60 of 60 with it.
- `IgniteTopologyValidatorGridSplitCacheTest`, where the problem was first
seen: the
partition-map timeout occurred in 9 of 15 local runs before the fix and in
none of 19 after.
Its other known failures are unrelated.
- A throwaway SQL check of the same scenario: `select count(*)` failed after
30 s on the
coordinator and on a third node without the fix, and returned in
milliseconds on all nodes
with it.
- `IgniteCachePartitionLossPolicySelfTest`,
`CachePartitionLostAfterSupplierHasLeftTest`,
`IgniteCachePartitionMapUpdateTest` and `GridExchangeFreeSwitchTest` pass.
- Not covered by a test: the coordinator owning a lost partition itself.
That path behaves as
before this change.
Thank you for submitting the pull request to the Apache Ignite.
In order to streamline the review of the contribution
we ask you to ensure the following steps have been taken:
### The Contribution Checklist
- [ ] There is a single JIRA ticket related to the pull request.
- [ ] The web-link to the pull request is attached to the JIRA ticket.
- [ ] The JIRA ticket has the _Patch Available_ state.
- [ ] The pull request body describes changes that have been made.
The description explains _WHAT_ and _WHY_ was made instead of _HOW_.
- [ ] The pull request title is treated as the final commit message.
The following pattern must be used: `IGNITE-XXXX Change summary` where
`XXXX` - number of JIRA issue.
- [ ] A reviewer has been mentioned through the JIRA comments
(see [the Maintainers
list](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers))
- [ ] The pull request has been checked by the Teamcity Bot and
the `green visa` attached to the JIRA ticket (see tab `PR Check` at [TC.Bot
- Instance 1](https://tcbot2.sbt-ignite-dev.ru/prs.html) or [TC.Bot - Instance
2](https://mtcga.gridgain.com/prs.html))
### Notes
- [How to
Contribute](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute)
- [Coding abbreviation
rules](https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules)
- [Coding
Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines)
- [Apache Ignite Teamcity
Bot](https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot)
If you need any help, please email [email protected] or ask anу advice
on http://asf.slack.com _#ignite_ channel.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
