[
https://issues.apache.org/jira/browse/SOLR-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15450041#comment-15450041
]
Shalin Shekhar Mangar commented on SOLR-9438:
---------------------------------------------
Beasting this test sometimes fails with nodes not recovering even after working
around SOLR-9440. I finally found the cause:
{code}
[beaster] 2> 232316 ERROR
(coreZkRegister-123-thread-1-processing-n:127.0.0.1:54683_
x:collection1_shard1_0_replica0 s:shard1_0 c:collection1 r:core_node7)
[n:127.0.0.1:54683_ c:collection1 s:shard1_0 r:core_node7
x:collection1_shard1_0_replica0] o.a.s.c.ZkContainer
:org.apache.solr.common.SolrException: Error getting leader from zk for shard
shard1_0
[beaster] 2> at
org.apache.solr.cloud.ZkController.getLeader(ZkController.java:994)
[beaster] 2> at
org.apache.solr.cloud.ZkController.register(ZkController.java:900)
[beaster] 2> at
org.apache.solr.cloud.ZkController.register(ZkController.java:843)
[beaster] 2> at
org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
[beaster] 2> at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[beaster] 2> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[beaster] 2> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[beaster] 2> at java.lang.Thread.run(Thread.java:745)
[beaster] 2> Caused by: org.apache.solr.common.SolrException: There is
conflicting information about the leader of shard: shard1_0 our state
says:http://127.0.0.1:54683/collection1_shard1_0_replica1/ but zookeeper
says:http://127.0.0.1:49547/collection1_shard1_0_replica1/
[beaster] 2> at
org.apache.solr.cloud.ZkController.getLeader(ZkController.java:975)
[beaster] 2> ... 7 more
{code}
The problem is that restarting the node (which assigns a new port number)
sometimes confuses the hell out of SolrCloud and then such nodes keep their old
port number in cluster state and never recover, can't elect leaders etc. I have
a suspicion that this behavior is intentional. I'll keep digging.
> Shard split can lose data
> -------------------------
>
> Key: SOLR-9438
> URL: https://issues.apache.org/jira/browse/SOLR-9438
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
> Affects Versions: 4.10.4, 5.5.2, 6.1
> Reporter: Shalin Shekhar Mangar
> Assignee: Shalin Shekhar Mangar
> Labels: difficulty-medium, impact-high
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-9438.patch, SOLR-9438.patch
>
>
> Solr’s shard split can lose documents if the parent/sub-shard leader is
> killed (or crashes) between the time that the new sub-shard replica is
> created and before it recovers. In such a case the slice has already been set
> to ‘recovery’ state, the sub-shard replica comes up, finds that no other
> replica is up, waits until the leader vote wait time and then proceeds to
> become the leader as well as publish itself as active. Once that happens the
> overseer seeing that all replicas of the sub-shard are now ‘active’, sets the
> parent slice as ‘inactive’ and the new sub-shard as ‘active’.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
