[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022985#comment-17022985 ] Zili Chen commented on FLINK-13417: --- [~chesnay] I agree that these are 2 separated discussion since by verifying above our current codebase works well on both zk34 and zk35. The problem is that we need to provide 2 bundled version so that users can choose the one satisfy their environment. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Assignee: Chesnay Schepler >Priority: Blocker > Labels: pull-request-available, usability > Fix For: 1.11.0 > > Time Spent: 20m > Remaining Estimate: 0h > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021908#comment-17021908 ] Chesnay Schepler commented on FLINK-13417: -- Making HA services pluggable is an orthogonal discussion to supporting multiple ZK versions imo. My current plan is to simple create 2 jars bundling ZK+ curator in flink-shaded, have one in lib, one in opt, and users pick whichever they need. This is pretty easy to set up (for us) and easy to understand (by users). It also provides an nice stepping stone for creating self-contained ZK 3.4/3.5 HA service implementations, since the dependencies are already there. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Assignee: Chesnay Schepler >Priority: Blocker > Labels: pull-request-available, usability > Fix For: 1.11.0 > > Time Spent: 20m > Remaining Estimate: 0h > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021811#comment-17021811 ] Zili Chen commented on FLINK-13417: --- [~sewen] per FLINK-10333 and other issues caused by HA Services we internally implemented a leader store based HA Services as described in FLINK-10333, which mainly uses a single truth source of HA operations. I like the idea we make HA Services pluggable so that we can load different strategy on demand. But it should have a well-designed interface. I'd prefer to discuss such changes public said in mailing list first, though. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Assignee: Chesnay Schepler >Priority: Blocker > Labels: pull-request-available, usability > Fix For: 1.11.0 > > Time Spent: 20m > Remaining Estimate: 0h > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021296#comment-17021296 ] Stephan Ewen commented on FLINK-13417: -- One thought [~trohrmann] and me had was making the HA Services truly pluggable and loading them from the plugins. Then we could have a ZK 3.4 and a ZK 3.5 version next to each other and specify in the config which one to use (by name / factory). This would be a rather involved change, though. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Assignee: Chesnay Schepler >Priority: Blocker > Labels: pull-request-available, usability > Fix For: 1.11.0 > > Time Spent: 20m > Remaining Estimate: 0h > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001048#comment-17001048
]
Till Rohrmann commented on FLINK-13417:
---
Unblocking {{1.10.0}} from this issue as it turns out that a new ZooKeeper
client cannot necessarily talk to an older ZooKeeper ensemble. Due to this, we
would break backwards compatibility with older ensembles if we upgrade now to
ZooKeeper 3.5.
Due to this fact, I believe that we have to offer different versions of Flink
(for ZooKeeper 3.4 and ZooKeeper 3.5). Maybe it is as easy as putting a
different ZooKeeper jar into {{lib}} but this needs to be properly tested.
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Assignee: Till Rohrmann
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.11.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1735#comment-1735 ] Zili Chen commented on FLINK-13417: --- > "if Flink would use ZooKeeper 3.5 features" [~trohrmann] yes. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Assignee: Till Rohrmann >Priority: Blocker > Labels: pull-request-available > Fix For: 1.10.0 > > Time Spent: 10m > Remaining Estimate: 0h > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1705#comment-1705 ] Till Rohrmann commented on FLINK-13417: --- cc [~liyu2][~GJL], escalated this issue to be a blocker for the 1.10.0 release. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Assignee: Till Rohrmann >Priority: Blocker > Labels: pull-request-available > Fix For: 1.10.0 > > Time Spent: 10m > Remaining Estimate: 0h > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1650#comment-1650
]
Till Rohrmann commented on FLINK-13417:
---
Hi [~tison], as far as I understood our previous conversion I thought that our
users would only need to upgrade their ZooKeeper cluster to 3.5 if Flink would
use ZooKeeper 3.5 features. Is this correct? If yes, then we must not yet use
{{CreateMode.CONTAINER}} and the likes.
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999287#comment-16999287 ] Till Rohrmann commented on FLINK-13417: --- For future reference: [Here|https://lists.apache.org/thread.html/be402a11bc986219eabd9dd8af507f36f49784d5400d0873e9ec0c2e%40%3Cdev.flink.apache.org%3E] is the link to the dev ML discussion about dropping vendor specific repositories. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > Fix For: 1.11.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999279#comment-16999279 ] Zili Chen commented on FLINK-13417: --- Thanks for reviving this thread [~sewen][~trohrmann]. One thing I'd like to emphasize again is that the bump to zk 3.5 will force user update version of their zookeeper ensemble. See also [this comment|https://issues.apache.org/jira/browse/FLINK-13417?focusedCommentId=16921386=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16921386] > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > Fix For: 1.11.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999263#comment-16999263 ] Till Rohrmann commented on FLINK-13417: --- Hi [~tison], thanks a lot for driving this effort. I would like to help with finalizing it so that we can include it in the upcoming 1.10 release. As a first step I would propose to drop the vendor specific repos. I'll start a discussion on dev for this. Then the version upgrade should be a bit easier. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > Fix For: 1.11.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992696#comment-16992696
]
Stephan Ewen commented on FLINK-13417:
--
[~tison] I think this one would be nice to get into 1.10 in the end, got some
pings about this issue personally as well.
The PR looks good (we can probably even remove the MapR specific profiles,
because that is left over from before the vendor-specific parts moved to
{{flink-shaded}}).
The most important question is testing against previous ZK versions. Did you
run tests against ZK 3.5 or ZK 3.4 when testing this?
To be confident about releasing this, I think it would be nice to have a test
setup that uses ZK 3.4.
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Major
> Fix For: 1.11.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967103#comment-16967103 ] David Judd commented on FLINK-13417: Flagging interest in this because ZK 3.5 also contains a fix for https://issues.apache.org/jira/browse/ZOOKEEPER-1576 - which we're running into & has the effect that (since we provide hostnames instead of IPs currently) the inability to reach a single Zookeeper host from the Flink job manager will bring it, and therefore the Flink cluster, down. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949076#comment-16949076 ] Zili Chen commented on FLINK-13417: --- Hi [~sewen], [here|https://github.com/apache/flink/pull/9840] is the verifying pr according to your suggestion. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944254#comment-16944254 ] Zili Chen commented on FLINK-13417: --- Another option to avoid dependency to bump curator is excluding zk from curator dependencies as done in pr of FLINK-10052. I am starting a new branch to verify it. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943594#comment-16943594 ] Zili Chen commented on FLINK-13417: --- Hello Stephan, the situation is actually flink-runtime uses zookeeper(flink-mesos used a wrapper version of curator's shared value which satisfy use through a utility way) so thay I think the way you proposed is worth to try. However, there's one prerequisite of this way. That's we need to bump curator-test to 4.x(related FLINK-10052) because curator-test 2.x uses zk 3.4 for starting server so that create container message from zk 2.5 client cannot be identified. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943572#comment-16943572 ] Stephan Ewen commented on FLINK-13417: -- Sorry for having been out of this thread for a while. If there is a way to avoid explicitly shaded modules (especially inside Flink) I believe that this is preferable. We are also trying hard to get other statically relocated libraries moved to flink-shaded. Is ZK used in more than one module (also outside flink-runtime)? In not, then it should be sufficient to just shade it inside the runtime package. If it is used in other places, is it used directly, or is it used through a runtime utility? If we can get to the later case, we could have ZK only shaded/relocated in flink-runtime. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942853#comment-16942853 ] Zili Chen commented on FLINK-13417: --- Run against master repo but it seems not trigger corn job. Could you help trigger corn job on zk-3,5-shaded branch [~trohrmann]? Although we should take several discussion about whether or not and if so, how to migrate from zk 3.4 to zk 3.5(since it enforces our users to bump their zk server version), we can concurrently verify that if there is any technical broker to do so. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942545#comment-16942545
]
Zili Chen commented on FLINK-13417:
---
For your information, I created {{flink-shaded-zookeeper}} under
{{flink-parent}} and pass all our test suites in this [pull
request|https://github.com/apache/flink/pull/9762] without modification outside
maven {{pom.xml}}.
I'm going to push a branch against master repo for running e2e tests and dig
out deeper.
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Major
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938381#comment-16938381 ] Zili Chen commented on FLINK-13417: --- Thanks for your information [~trohrmann]. FYI I open a testing [pull request|https://github.com/apache/flink/pull/9762] for shading and bumping ZK. I'll really appreciate it if you can help with early review. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932233#comment-16932233
]
Till Rohrmann commented on FLINK-13417:
---
Try {{mvn dependency:tree}} on Flink's root directory. That should show you all
dependencies.
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Major
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931424#comment-16931424 ] TisonKun commented on FLINK-13417: -- [~till.rohrmann] thanks for your advice. For the tools issue I believe I fully built Flink before running the tool but it generates incorrect NOTICE-binary. Anyway, I will create a separated issue with details when I meet it next time. For the step "replace all Flink dependencies on ZooKeeper with this one", I'd like to know whether there is a good way to list out all Flink dependencies on ZooKeeper. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931415#comment-16931415
]
Till Rohrmann commented on FLINK-13417:
---
[~Tison] {{tools/releasing/collect_license_files.sh}} should generate the
license file which should become the new {{NOTICE-binary}} and the new
{{licenses-binary}} directory. The important bit is to run it in
{{build-target}} after fully building Flink:
{code}
Step 1: Rebuild flink
Step 2: Run 'tools/releasing/collect_license_files.sh build-target'
This extracts all the licensing files from the distribution, and puts them in
'licenses-output'.
If the build-target symlink does not exist after building flink, point the
tool to 'flink-dist/target/flink--bin/flink-' instead.
Step 3: Replace existing licensing
Delete NOTICE-binary and the entire licenses-binary directory.
Copy the contents in 'licenses-output' into the root directory of the Flink
project.
Step 4: Remember to commit the changes!
{code}
1. You could start with creating a {{flink-shaded-zookeeper}} module but
eventually I would suggest to add it to
[flink-shaded|https://github.com/apache/flink-shaded].
2. I guess we should replace all Flink dependencies on ZooKeeper with this one
then.
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Major
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931131#comment-16931131
]
TisonKun commented on FLINK-13417:
--
Updates: Manually apply the diff and go through. Maybe there is something
non-deterministic in {{tools.releasing/collect_license_files.sh}}.
Travis deadlock on {{KafkaTestBase}} because ZK AuthFailed, full log is here
https://api.travis-ci.org/v3/job/585870154/log.txt
[~StephanEwen] yes I think it is reasonable that shade Flink's ZK as you
proposed. Otherwise we have to twist our tests quite a bit to let them work.
I'd like to give it a try to see the effect. However, I'm not so familiar with
the procedure so here I'd like to describe what I want to do first.
1. create a sub-module {{flink-shaded-zookeeper}} as what we do for
{{flink-shaded-curator}}. Here I don't know how to generate the NOTICE file.
2. replace {{zookeeper}} dependencies in Flink scope with
{{flink-shaded-zookeeper}}. Here I don't know whether there is an automatic way
to list which dependencies should be replaced.
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Major
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930377#comment-16930377 ] Aljoscha Krettek commented on FLINK-13417: -- I moved this from "Blocker" to "Major" priority since I don't think a version upgrade or new feature should be a blocker for a release. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928499#comment-16928499 ] Stephan Ewen commented on FLINK-13417: -- @Tison would it be safer to shade Flink's ZK and then let the HBase client bring its own version / config / everything? I guess in a proper setup, it should work anyways due to inverted class loading (HBase connector uses its own dependency in the user jar, not the Flink ZK dependency), but not for tests. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Blocker > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928459#comment-16928459
]
TisonKun commented on FLINK-13417:
--
Hi [~till.rohrmann] I met a NOTICE-binary issue and cannot resolve it follow
the guide output. So I'd like to reach you for help.
With the fix mentioned above we don't fail on the same cause now. But after I
do a rebase on master I encounter a failure message as below
{noformat}
> Apache ZooKeeper - Server
> Copyright 2008-2019 The Apache Software Foundation
>
> Apache ZooKeeper - Jute
> Copyright 2008-2019 The Apache Software Foundation
>
7922a7929,7937
>
> Apache ZooKeeper - Server
> Copyright 2008-2019 The Apache Software Foundation
>
> Apache ZooKeeper - Jute
> Copyright 2008-2019 The Apache Software Foundation
>
> Apache Yetus - Audience Annotations
> Copyright 2015-2017 The Apache Software Foundation
==
ERROR: binary licensing is out-of-date.
Please update NOTICE-binary and licenses-binary:
Step 1: Rebuild flink
Step 2: Run 'tools/releasing/collect_license_files.sh build-target'
This extracts all the licensing files from the distribution, and puts them in
'licenses-output'.
If the build-target symlink does not exist after building flink, point the
tool to 'flink-dist/target/flink--bin/flink-' instead.
Step 3: Replace existing licensing
Delete NOTICE-binary and the entire licenses-binary directory.
Copy the contents in 'licenses-output' into the root directory of the Flink
project.
Step 4: Remember to commit the changes!
==
{noformat}
full log is here https://api.travis-ci.org/v3/job/584054593/log.txt
so I follow the guide, regenerate the NOTICE file and do the replacement.
However, I met another NOTICE failure
full log is here https://api.travis-ci.org/v3/job/584027857/log.txt
(do not attach the second output since it is quite a bit long)
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Blocker
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928316#comment-16928316
]
TisonKun commented on FLINK-13417:
--
[~StephanEwen] I figured out that it is because since 3.5.3 ZooKeeper default
disable four letter words which HBase uses for waiting for server up. We can
set the property in Flink test scope to enable four letter words and workaround
this issue. Have sent the fix and still digging whether there are other issues.
references:
[1] https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_4lw
[2]
https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_clusterOptions
(look up {{zookeeper.4lw.commands.whitelist}})
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Blocker
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16925583#comment-16925583 ] Stephan Ewen commented on FLINK-13417: -- @Tison is that a mismatch between ZK versions now? That the HBase tests assume a specific version? Do we need to shade the ZK client either in Flink or in the HBase tests? > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Blocker > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922970#comment-16922970
]
TisonKun commented on FLINK-13417:
--
Yes [~till.rohrmann]. I locally build with zk 3.5 and no compile error reported
while I fired CI it passed almost builds, see also
https://travis-ci.org/TisonKun/flink/builds/580757901
which reported failures on {{HBaseConnectorITCase}} when started HBase
MiniCluster when started MiniZooKeeperCluster. It seems like a problem of
testing class implementation.
Maybe [~carp84] can provide some inputs here.
{code:java}
java.io.IOException: Waiting for startup of standalone server
at
org.apache.hadoop.hbase.zookeeper.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:261)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniZKCluster(HBaseTestingUtility.java:814)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniZKCluster(HBaseTestingUtility.java:784)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1041)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:917)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:899)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:881)
at
org.apache.flink.addons.hbase.util.HBaseTestingClusterAutoStarter.setUp(HBaseTestingClusterAutoStarter.java:147)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
{code}
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Blocker
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922289#comment-16922289
]
Till Rohrmann commented on FLINK-13417:
---
Ok, so you are saying that the upgrade from ZooKeeper 3.4 to 3.5 won't cause
any incompatibilities. Only if we start using ZooKeeper 3.5 features such as
the {{CreateMode.CONTAINER}} our users would need to upgrade to ZooKeeper 3.5
as well. Is this correct?
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Blocker
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921386#comment-16921386
]
TisonKun commented on FLINK-13417:
--
[~till.rohrmann]
Here is [the
thread|https://lists.apache.org/x/thread.html/a35ef4da607121ac3621f7eee01b377b4d205b6bf676357bd25d949f@%3Cuser.zookeeper.apache.org%3E]
as well as the upgrade FAQ
[page|https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ].
It is announced no intended or known break changes if we upgrade zk to 3.5 and
require quorums running on 3.5. But if we upgrade to zk 3.5, since we would
like to make used of {{CreateMode.CONTAINER}} the packet won't be identified by
a 3.4 server and will fail with {{KeeperException.UnimplementedException}}.
Simply sum up, the upgrade should not cost any changes in FLINK side but
required our users running FLINK against zk 3.5 servers. Unless we add an
switch to opt-in and opt-out, with manually fallback the use of new features
introduced in 3.5.
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Blocker
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921268#comment-16921268 ] Till Rohrmann commented on FLINK-13417: --- Any progress on this issue [~Tison]? > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Blocker > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915745#comment-16915745 ] TisonKun commented on FLINK-13417: -- I started a thread in ZooKeeper community to see if there are known migration issues and try to handle them for our smooth upgrade. FYI I have been using 3.5.x client communicate with 3.4.x servers for a while and seen no migration issues so far. Bump Curator version to 4.2.0 is also a requirement for FLINK-10052. I think it is orthogonal with bump ZooKeeper. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Blocker > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915740#comment-16915740
]
Till Rohrmann commented on FLINK-13417:
---
Are we intending to drop support for ZooKeeper {{3.4}}? This might break
existing setups where users have a ZooKeeper 3.4 quorum running. However, I'm
not sure how widespread the usage of {{3.4}} still is.
I assume that we should then also bump the curator version to {{4.2.0}}.
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Blocker
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905334#comment-16905334
]
Stephan Ewen commented on FLINK-13417:
--
The {{flink-jepsen}} readme has some pointers to get started:
https://github.com/apache/flink/tree/master/flink-jepsen
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Blocker
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898470#comment-16898470
]
TisonKun commented on FLINK-13417:
--
[~StephanEwen] I'm glad to help. Let me see what I can do for it and I'd like
to file a subtask of this thread once I got one.
Could you show me where is a good start for making use of {{flink-jepsen}}?
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Blocker
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897168#comment-16897168
]
Stephan Ewen commented on FLINK-13417:
--
agree that we need to test this carefully.
[~Tison] would you be interested in helping out with this? Maybe use the
{{flink-jepsen}} tests (which implement some master failover scenario with ZK)
to verify this?
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Blocker
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16896984#comment-16896984
]
TisonKun commented on FLINK-13417:
--
A proper fix to FLINK-12006 depends on using the {{CreateMode#CONTAINER}} which
was introduced in 3.5.x.
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Blocker
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16895170#comment-16895170 ] Nico Kruber commented on FLINK-13417: - FYI: since 3.5.5 is the first stable version in the 3.5.x series[1], we should actually take this, not any older 3.5.x [1] https://zookeeper.apache.org/releases.html > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16892464#comment-16892464 ] TisonKun commented on FLINK-13417: -- Nice proposal [~knaufk]! IIRC several runtime issues could be solved with zookeeper CONTAINER znode mode that introduced in 3.5.x ZooKeeper has just released the production ready version 3.5.5 so I generally vote for this issue. Keep in mind that it would(in fact, I think it must) cause some subtle issues so it should be fully tested. cc [~till.rohrmann], I would try to link those runtime issues later. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Reporter: Konstantin Knauf >Priority: Major > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
