[
https://issues.apache.org/jira/browse/IGNITE-28952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Davydov updated IGNITE-28952:
------------------------------------
Description:
MdcTopologyValidator has two modes and the number of data centers picks one:
with
an EVEN DC count a segment stays writable while it sees the main DC, with an ODD
one while it sees a majority of the DC set. The ducktest fixture only ever
built a
two DC cluster, so only the first mode is covered.
Let MdcCluster span an arbitrary DC set, compile the cache parameters from it,
and
cut several cross-DC links apart in one round-trip. On top of that, three
scenarios
over three DCs: one DC isolated, a three way split, and a DC lost outright.
from the two others, parametrized over which DC is isolated; a three way
split in which
every segment goes read-only; and data centers lost one by one, writable
while a majority
survives and read-only below it. Every partition owns one copy per DC, so
every segment
still serves every read - that is what separates the read assertions from
the write ones.
- Framework unit checks for the DC-count dependent parts: validator mode
selection, backup
arithmetic, the impairment mesh and its per-pair overrides, and the
per-node command
compilation of single and multi-link partitions.
was:
The MDC test fixture hardcodes two things the MDC feature itself does not
require: that every cache carries an MdcTopologyValidator, and that the cluster
spans exactly two data centers.
1. Cache level topology validator
The MDC test applications always configure the cache with MdcTopologyValidator
(MdcCacheAwareApplication), so the scenarios only run against a validator
configured per
cache.
2. Number of data centers
MdcTopologyValidator has two modes, and the number of data centers selects
which one
applies: an even set is validated against a main datacenter, an odd set
against a majority
of the set. The fixture only ever exercised the even one - DCS = (DC1, DC2)
is a module
level constant threaded through the whole of MdcCluster, verify_split_brain()
assumes
exactly two half-rings, and cross_dc_network() impairs a single DC pair.
Changes:
- MdcCacheAwareApplication: new 'topologyValidator' parameter, default true.
When false,
the cache is created without the cache level validator and the
mainDc/datacenters
parameters are not required.
- MdcCluster: new 'mdc_cache_topology_validator' global (default true)
applied to every
cache the fixture creates.
- MdcCluster takes a 'dcs' sequence (DCS_2 by default, DCS_3 provided).
mdc_topology_params()
derives the validator mode from the DC count and emits exactly one of
mainDc/datacenters
- MdcTopologyValidator.checkConfiguration() rejects a main datacenter
alongside an odd DC
set - plus the dcsNum MdcAffinityBackupFilter needs. min_backups() derives
the smallest
backup count giving every DC one copy of every partition, which is 2 for
three DCs.
- verify_split_brain() generalizes into verify_segments() over an arbitrary
segmentation;
verify_half_ring_healthy() becomes verify_segment_healthy().
cross_dc_network() impairs
the whole cross-DC mesh and takes per-pair overrides for asymmetric links.
- NetworkGroupManager: new
enable_network_partitions()/disable_network_partitions() cut
several links at once. Every chain a node takes part in is installed by the
same SSH
round-trip, because rolling a multi-way split out link by link would
briefly present the
cluster with a different, valid segmentation it would legitimately react
to. The pairwise
methods delegate to them.
- New scenarios in mdc/majority_partition_test.py (9 nodes each): a minority
DC isolated
from the two others, parametrized over which DC is isolated; a three way
split in which
every segment goes read-only; and data centers lost one by one, writable
while a majority
survives and read-only below it. Every partition owns one copy per DC, so
every segment
still serves every read - that is what separates the read assertions from
the write ones.
- Framework unit checks for the DC-count dependent parts: validator mode
selection, backup
arithmetic, the impairment mesh and its per-pair overrides, and the
per-node command
compilation of single and multi-link partitions.
> [ducktests] MDC tests for majority based topology validation
> ------------------------------------------------------------
>
> Key: IGNITE-28952
> URL: https://issues.apache.org/jira/browse/IGNITE-28952
> Project: Ignite
> Issue Type: Task
> Reporter: Maksim Davydov
> Assignee: Maksim Davydov
> Priority: Minor
> Labels: IEP-140, ducktests, ise
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> MdcTopologyValidator has two modes and the number of data centers picks one:
> with
> an EVEN DC count a segment stays writable while it sees the main DC, with an
> ODD
> one while it sees a majority of the DC set. The ducktest fixture only ever
> built a
> two DC cluster, so only the first mode is covered.
> Let MdcCluster span an arbitrary DC set, compile the cache parameters from
> it, and
> cut several cross-DC links apart in one round-trip. On top of that, three
> scenarios
> over three DCs: one DC isolated, a three way split, and a DC lost outright.
> from the two others, parametrized over which DC is isolated; a three way
> split in which
> every segment goes read-only; and data centers lost one by one, writable
> while a majority
> survives and read-only below it. Every partition owns one copy per DC, so
> every segment
> still serves every read - that is what separates the read assertions from
> the write ones.
> - Framework unit checks for the DC-count dependent parts: validator mode
> selection, backup
> arithmetic, the impairment mesh and its per-pair overrides, and the
> per-node command
> compilation of single and multi-link partitions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
