[
https://issues.apache.org/jira/browse/SPARK-20884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-20884.
-------------------------------
Resolution: Won't Fix
We can't merge this change, because Curator 2.8 won't work with Curator 2.6
environments, which include Hadoop 2.6/2.7 envs, which are still supported.
Much later we could update this, basically when Spark 3 moves on to Hadoop 3
(which uses Curator 2.12)
> Spark' masters will be both standby due to the bug of curator
> --------------------------------------------------------------
>
> Key: SPARK-20884
> URL: https://issues.apache.org/jira/browse/SPARK-20884
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.1.1
> Reporter: liuzhaokun
>
> I built a cluster with two masters and three workers.When there is a switch
> of master's state,two masters are both standby in a long period.There are
> some ERROR in master's logfile " ERROR CuratorFrameworkImpl: Background
> exception was not retry-able or retry gave up
> java.lang.IllegalArgumentException: Path must start with / character
> at org.apache.curator.utils.PathUtils.validatePath(PathUtils.java:53)
> at org.apache.curator.utils.ZKPaths.getNodeFromPath(ZKPaths.java:56)
> at
> org.apache.curator.framework.recipes.leader.LeaderLatch.checkLeadership(LeaderLatch.java:421)
> at
> org.apache.curator.framework.recipes.leader.LeaderLatch.access$500(LeaderLatch.java:60)
> at
> org.apache.curator.framework.recipes.leader.LeaderLatch$6.processResult(LeaderLatch.java:478)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:686)
> at
> org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:485)
> at
> org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:587)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)"
> it can be related to https://issues.apache.org/jira/browse/CURATOR-168.
> So I think the version of curator in spark should be 2.8.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
