maksaska opened a new pull request, #13625:
URL: https://github.com/apache/ignite/pull/13625
Adds a base for JUnit tests that cut the network between the data centers
(DCs) of one cluster, and a first test that checks the topology validator on
it. Test code only.
Why. The existing template, IgniteCacheTopologySplitAbstractTest, splits a
cluster into two halves and waits for one exact topology version without a
timeout. A split that ends in a different topology hangs the test until it is
killed. Its heal delivers the messages held during the split, which replays one
side's view of the cluster on the other side. Multi-DC tests also need splits
by DC and more than two sides.
MdcTopologySplitAbstractTest extends the template without changing it. It
reuses the template's SplitTcpDiscoverySpi (discovery connections fail with a
socket timeout) and TestRecordingCommunicationSpi (communication messages are
held).
- Each DC has two servers and one client; the DC is set with
IGNITE_DATA_CENTER_ID. A client's IP finder lists only its own DC's servers, so
the client stays on its DC's side of a split.
- splitInto(sides) splits the DCs into two or more sides; split(dcs) cuts
the given DCs off the rest. Communication between sides is held first, then
discovery is cut. Both decide by the DC of each end.
- The split waits up to 30 s until every node sees exactly its own side and
has finished its last exchange. Otherwise the test fails and logs what every
node sees.
- heal(dcs) restarts every side but one and drops the held messages. The
restarted nodes rejoin and rebalance from the side that stayed up. It waits for
the whole cluster with the same limit.
- Helpers for the tests that build on it:
- a cache configuration with one copy of each partition per DC;
- majority and main-DC validators;
- assertWriteRejected, which accepts only the topology validator's own
rejection and searches the cause chain, since an implicit transaction wraps it;
- assertDataInEveryDc.
- blockMessage returns false here, and no test in this change overrides it.
It exists for a follow-up test of transactions cut by a split, which holds one
message during the split. The communication SPI takes a single filter, so a
subclass's own filter would replace the split's filter or be replaced by it.
The base combines both instead.
MdcDcIsolationTest writes to an atomic and a transactional cache before the
split, checks writes and reads on each side, heals, then checks that every DC
has the same data and that idle_verify finds no conflicts:
- 3 DCs, majority validator, DC3 cut off: DC1 and DC2 write, DC3 rejects
writes and serves reads.
- The same with DC1 cut off (DC2 writes), so that no DC is privileged.
- 3 DCs split three ways: no side writes, every side reads the data written
before the split.
- 2 DCs with DC1 as the main one: DC2 cut off rejects writes and serves
reads.
The test is registered in IgniteTopologyValidatorTestSuite.
Testing.
- Each check was shown to fail first:
- without the validator, the cut-off DC accepts writes;
- with a main-DC validator instead of majority, the DC1 side of the
three-way split accepts writes;
- with a split that blocks nothing, the wait fails after 30 s instead of
hanging.
- The class passed 40 runs in a row with no failure or hang: all four
methods, about 2 minutes per run.
- Each split settled in about 10 s, which is the failure detection time. The
slowest took 19.6 s once.
Thank you for submitting the pull request to the Apache Ignite.
In order to streamline the review of the contribution
we ask you to ensure the following steps have been taken:
### The Contribution Checklist
- [ ] There is a single JIRA ticket related to the pull request.
- [ ] The web-link to the pull request is attached to the JIRA ticket.
- [ ] The JIRA ticket has the _Patch Available_ state.
- [ ] The pull request body describes changes that have been made.
The description explains _WHAT_ and _WHY_ was made instead of _HOW_.
- [ ] The pull request title is treated as the final commit message.
The following pattern must be used: `IGNITE-XXXX Change summary` where
`XXXX` - number of JIRA issue.
- [ ] A reviewer has been mentioned through the JIRA comments
(see [the Maintainers
list](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers))
- [ ] The pull request has been checked by the Teamcity Bot and
the `green visa` attached to the JIRA ticket (see tab `PR Check` at [TC.Bot
- Instance 1](https://tcbot2.sbt-ignite-dev.ru/prs.html) or [TC.Bot - Instance
2](https://mtcga.gridgain.com/prs.html))
### Notes
- [How to
Contribute](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute)
- [Coding abbreviation
rules](https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules)
- [Coding
Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines)
- [Apache Ignite Teamcity
Bot](https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot)
If you need any help, please email [email protected] or ask anу advice
on http://asf.slack.com _#ignite_ channel.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
