[
https://issues.apache.org/jira/browse/IGNITE-10058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16724833#comment-16724833
]
Pavel Pereslegin edited comment on IGNITE-10058 at 12/19/18 9:59 AM:
---------------------------------------------------------------------
Hello [~ilantukh],
Thank you very much, I reworked patch according your suggestion, but with a few
notes.
_> {color:#707070}Modify GridDhtPartitionTopologyImpl.resetLostPartitions(...)
- it should reset update counters to 0 only if at the moment when the method
was called there was at least one partition owner{color}_
Each node resets lost partitions before send single message to coordinator,
coordinator resets lost partitions after update partition map from single
messages. So I moved {{resetLostPartitions()}} method call to exchange init
(otherwise it observes states from updated partition map).
Also I noticed that after this change (don't reset update counters if there
are no owners) partition size validation fails with warning like "_Partition
states validation has failed for group: default. Partitions cache sizes are
inconsistent for Part 1: [cache.IndexingCachePartitionLossPolicySelfTest0=1
cache.IndexingCachePartitionLossPolicySelfTest3=0 ] ..._"
({{IgniteCachePartitionLossPolicySelfTest#testReadWriteSafe}}). Is this an
issue?
_> {color:#707070}Also, add special logic for the case when all LOST partitions
already have update counter 0 - transfer state to OWNING only on affinity
nodes{color}._
I didn't quite understand how to simply implement this validation on each
node, because only coordinator has all counters from all nodes (when received
all single messages).
The main use case, described in this ticket (duplicate owners after reset),
fixed by checking the rebalancing state ({{checkRebalanceState}}) after
resetting lost partitions on the coordinator.
was (Author: xtern):
Hello [~ilantukh],
Thank you very much, I reworked patch according your suggestion, but with a few
notes.
_> {color:#707070}Modify GridDhtPartitionTopologyImpl.resetLostPartitions(...)
- it should reset update counters to 0 only if at the moment when the method
was called there was at least one partition owner{color}_
Each node resets lost partitions before send single message to coordinator,
coordinator resets lost partitions after update partition map from single
messages. So I moved {{resetLostPartitions()}} method call to exchange init
future (otherwise it observes states from updated partition map).
Also I noticed that after this change (don't reset update counters if there are
no owners) partition size validation fails with warning like "_Partition states
validation has failed for group: default. Partitions cache sizes are
inconsistent for Part 1: [cache.IndexingCachePartitionLossPolicySelfTest0=1
cache.IndexingCachePartitionLossPolicySelfTest3=0 ] ..._"
({{IgniteCachePartitionLossPolicySelfTest#testReadWriteSafe}}). Is this an
issue?
_> {color:#707070}Also, add special logic for the case when all LOST partitions
already have update counter 0 - transfer state to OWNING only on affinity
nodes{color}._
I didn't quite understand how to simply implement this validation on each node,
because only coordinator has all counters from all nodes (when received all
single messages).
The main use case, described in this ticket (duplicate owners after reset),
fixed by checking the rebalancing state ({{checkRebalanceState}}) after
resetting lost partitions on the coordinator.
> resetLostPartitions() leaves an additional copy of a partition in the cluster
> -----------------------------------------------------------------------------
>
> Key: IGNITE-10058
> URL: https://issues.apache.org/jira/browse/IGNITE-10058
> Project: Ignite
> Issue Type: Bug
> Reporter: Stanislav Lukyanov
> Assignee: Pavel Pereslegin
> Priority: Major
> Fix For: 2.8
>
>
> If there are several copies of a LOST partition, resetLostPartitions() will
> leave all of them in the cluster as OWNING.
> Scenario:
> 1) Start 4 nodes, a cache with backups=0 and READ_WRITE_SAFE, fill the cache
> 2) Stop one node - some partitions are recreated on the remaining nodes as
> LOST
> 3) Start one node - the LOST partitions are being rebalanced to the new node
> from the existing ones
> 4) Wait for rebalance to complete
> 5) Call resetLostPartitions()
> After that the partitions that were LOST become OWNING on all nodes that had
> them. Eviction of these partitions doesn't start.
> Need to correctly evict additional copies of LOST partitions either after
> rebalance on step 4 or after resetLostPartitions() call on step 5.
> Current resetLostPartitions() implementation does call checkEvictions(), but
> the ready affinity assignment contains several nodes per partition for some
> reason.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
