[
https://issues.apache.org/jira/browse/FLINK-5996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17273789#comment-17273789
]
Till Rohrmann commented on FLINK-5996:
--------------------------------------
Flink's contract is that if the {{ZooKeeperClient}} loses connection to the
ZooKeeper cluster, which is effectively the same as the connection being
SUSPENDED, then Flink will assume that it is no longer the leader. The reason
for this is that we cannot learn about any changes which happen in the
meantime. If Flink wouldn't do this, then you might end up in a situation where
you have multiple leaders and thereby a split brain situation. There is also a
ticket FLINK-10052 which tries to make this behaviour configurable.
Now what you should look into is why Flink loses the connection
[~YordanPavlov]. One reason could be that it could only connect to a single
ZooKeeper server and this one fails later. But the fact that you have a quorum
does not guarantee that the client can always connect to any of the members of
the quorum.
> Jobmanager HA should not crash on lost ZK node
> ----------------------------------------------
>
> Key: FLINK-5996
> URL: https://issues.apache.org/jira/browse/FLINK-5996
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
> Affects Versions: 1.2.0
> Reporter: Gyula Fora
> Assignee: Dominik Wosiński
> Priority: Major
>
> Even if there are multiple zk hosts configured the jobmanager crashes if one
> of them is lost:
> org.apache.flink.shaded.org.apache.curator.CuratorConnectionLossException:
> KeeperErrorCode = ConnectionLoss
> at
> org.apache.flink.shaded.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:197)
> at
> org.apache.flink.shaded.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:87)
> at
> org.apache.flink.shaded.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:115)
> at
> org.apache.flink.shaded.org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:477)
> at
> org.apache.flink.shaded.org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:302)
> at
> org.apache.flink.shaded.org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:291)
> at
> org.apache.flink.shaded.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
> at
> org.apache.flink.shaded.org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
> at
> org.apache.flink.shaded.org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
> at
> org.apache.flink.shaded.org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
> at
> org.apache.flink.shaded.org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
> at
> org.apache.flink.shaded.org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
> at
> org.apache.flink.shaded.org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
> at
> org.apache.flink.shaded.org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
> We should have some retry logic there
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
