[jira] [Commented] (FLINK-6805) Flink Cassandra connector dependency on Netty disagrees with Flink

2017-10-15 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205193#comment-16205193
 ] 

ASF GitHub Bot commented on FLINK-6805:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/4545


> Flink Cassandra connector dependency on Netty disagrees with Flink
> --
>
> Key: FLINK-6805
> URL: https://issues.apache.org/jira/browse/FLINK-6805
> Project: Flink
>  Issue Type: Bug
>  Components: Cassandra Connector
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Shannon Carey
>Assignee: Michael Fong
>Priority: Blocker
> Fix For: 1.4.0
>
>
> The Flink Cassandra connector has a dependency on Netty libraries (via 
> promotion of transitive dependencies by the Maven shade plugin) at version 
> 4.0.33.Final, which disagrees with the version included in Flink of 
> 4.0.27.Final which is included & managed by the parent POM via dependency on 
> netty-all.
> Due to use of netty-all, the dependency management doesn't take effect on the 
> individual libraries such as netty-handler, netty-codec, etc.
> I suggest that dependency management of Netty should be added for all Netty 
> libraries individually (netty-handler, etc.) so that all Flink modules use 
> the same version, and similarly I suggest that exclusions be added to the 
> quickstart example POM for the individual Netty libraries so that fat JARs 
> don't include conflicting versions of Netty.
> It seems like this problem started when FLINK-6084 was implemented: 
> transitive dependencies of the flink-connector-cassandra were previously 
> omitted, and now that they are included we must make sure that they agree 
> with the Flink distribution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6805) Flink Cassandra connector dependency on Netty disagrees with Flink

2017-08-22 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136648#comment-16136648
 ] 

ASF GitHub Bot commented on FLINK-6805:
---

Github user zentol commented on the issue:

https://github.com/apache/flink/pull/4545
  
looks good to me, will merge this later.


> Flink Cassandra connector dependency on Netty disagrees with Flink
> --
>
> Key: FLINK-6805
> URL: https://issues.apache.org/jira/browse/FLINK-6805
> Project: Flink
>  Issue Type: Bug
>  Components: Cassandra Connector
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Shannon Carey
>Assignee: Michael Fong
> Fix For: 1.4.0
>
>
> The Flink Cassandra connector has a dependency on Netty libraries (via 
> promotion of transitive dependencies by the Maven shade plugin) at version 
> 4.0.33.Final, which disagrees with the version included in Flink of 
> 4.0.27.Final which is included & managed by the parent POM via dependency on 
> netty-all.
> Due to use of netty-all, the dependency management doesn't take effect on the 
> individual libraries such as netty-handler, netty-codec, etc.
> I suggest that dependency management of Netty should be added for all Netty 
> libraries individually (netty-handler, etc.) so that all Flink modules use 
> the same version, and similarly I suggest that exclusions be added to the 
> quickstart example POM for the individual Netty libraries so that fat JARs 
> don't include conflicting versions of Netty.
> It seems like this problem started when FLINK-6084 was implemented: 
> transitive dependencies of the flink-connector-cassandra were previously 
> omitted, and now that they are included we must make sure that they agree 
> with the Flink distribution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6805) Flink Cassandra connector dependency on Netty disagrees with Flink

2017-08-16 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128581#comment-16128581
 ] 

ASF GitHub Bot commented on FLINK-6805:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4545#discussion_r133407236
  
--- Diff: flink-connectors/flink-connector-cassandra/pom.xml ---
@@ -83,6 +85,10 @@ under the License.

com.google.inject.**



+   
+   
io.netty
+   
org.apache.flink.shaded.netty4.io.netty
--- End diff --

1) We haven't written them down anywhere; but there are examples on how the 
pattern should look like in many modules. (Granted, there are some relocations 
that don't follow the points I made; standardizing them is a work-in-progress) 
I would generally refer to existing relocations instead of some document that 
is bound to be out-dated at some point.
2) Currently we shade dependencies that already caused conflicts.
3) Yes, but there is no better solution. While it is possible to re-use 
some dependencies from Flink this tends to be a maintaining nightmare, as an 
upgrade to a dependency can suddenly cause issues in another part. The netty 
shading is a perfect example of this: we replaced the netty dependency and now 
the connector is broken.


> Flink Cassandra connector dependency on Netty disagrees with Flink
> --
>
> Key: FLINK-6805
> URL: https://issues.apache.org/jira/browse/FLINK-6805
> Project: Flink
>  Issue Type: Bug
>  Components: Cassandra Connector
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Shannon Carey
>Assignee: Michael Fong
> Fix For: 1.4.0
>
>
> The Flink Cassandra connector has a dependency on Netty libraries (via 
> promotion of transitive dependencies by the Maven shade plugin) at version 
> 4.0.33.Final, which disagrees with the version included in Flink of 
> 4.0.27.Final which is included & managed by the parent POM via dependency on 
> netty-all.
> Due to use of netty-all, the dependency management doesn't take effect on the 
> individual libraries such as netty-handler, netty-codec, etc.
> I suggest that dependency management of Netty should be added for all Netty 
> libraries individually (netty-handler, etc.) so that all Flink modules use 
> the same version, and similarly I suggest that exclusions be added to the 
> quickstart example POM for the individual Netty libraries so that fat JARs 
> don't include conflicting versions of Netty.
> It seems like this problem started when FLINK-6084 was implemented: 
> transitive dependencies of the flink-connector-cassandra were previously 
> omitted, and now that they are included we must make sure that they agree 
> with the Flink distribution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6805) Flink Cassandra connector dependency on Netty disagrees with Flink

2017-08-16 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128565#comment-16128565
 ] 

ASF GitHub Bot commented on FLINK-6805:
---

Github user mcfongtw commented on a diff in the pull request:

https://github.com/apache/flink/pull/4545#discussion_r133404126
  
--- Diff: flink-connectors/flink-connector-cassandra/pom.xml ---
@@ -83,6 +85,10 @@ under the License.

com.google.inject.**



+   
+   
io.netty
+   
org.apache.flink.shaded.netty4.io.netty
--- End diff --

Thanks for your review and reply! I see your points now, and I still have a 
few questions to the shading approach though:
1. There seems to be some policies for making shadedPattern for different 
scenarios. Would there be any external documents that possibly address these 
rules?
2. It is obvious to shade a *direct dependency, since we need it to be free 
of class clashes. How would we decide which *indirect dependencies to be shaded?
3. The benefit of shading is clear, but wouldn't it increase the jar file 
size if there are more dependencies that need to be managed later on - also the 
effort of managing the including/relocation rules.

I will update another version of PR later. Thanks!


> Flink Cassandra connector dependency on Netty disagrees with Flink
> --
>
> Key: FLINK-6805
> URL: https://issues.apache.org/jira/browse/FLINK-6805
> Project: Flink
>  Issue Type: Bug
>  Components: Cassandra Connector
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Shannon Carey
>Assignee: Michael Fong
> Fix For: 1.4.0
>
>
> The Flink Cassandra connector has a dependency on Netty libraries (via 
> promotion of transitive dependencies by the Maven shade plugin) at version 
> 4.0.33.Final, which disagrees with the version included in Flink of 
> 4.0.27.Final which is included & managed by the parent POM via dependency on 
> netty-all.
> Due to use of netty-all, the dependency management doesn't take effect on the 
> individual libraries such as netty-handler, netty-codec, etc.
> I suggest that dependency management of Netty should be added for all Netty 
> libraries individually (netty-handler, etc.) so that all Flink modules use 
> the same version, and similarly I suggest that exclusions be added to the 
> quickstart example POM for the individual Netty libraries so that fat JARs 
> don't include conflicting versions of Netty.
> It seems like this problem started when FLINK-6084 was implemented: 
> transitive dependencies of the flink-connector-cassandra were previously 
> omitted, and now that they are included we must make sure that they agree 
> with the Flink distribution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6805) Flink Cassandra connector dependency on Netty disagrees with Flink

2017-08-16 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128540#comment-16128540
 ] 

ASF GitHub Bot commented on FLINK-6805:
---

Github user mcfongtw commented on a diff in the pull request:

https://github.com/apache/flink/pull/4545#discussion_r13339
  
--- Diff: flink-connectors/flink-connector-cassandra/pom.xml ---
@@ -39,6 +39,7 @@ under the License.

2.2.5
3.0.0
+   4.0.27.Final
--- End diff --

Good call! It's not used currently. I will remove it .


> Flink Cassandra connector dependency on Netty disagrees with Flink
> --
>
> Key: FLINK-6805
> URL: https://issues.apache.org/jira/browse/FLINK-6805
> Project: Flink
>  Issue Type: Bug
>  Components: Cassandra Connector
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Shannon Carey
>Assignee: Michael Fong
> Fix For: 1.4.0
>
>
> The Flink Cassandra connector has a dependency on Netty libraries (via 
> promotion of transitive dependencies by the Maven shade plugin) at version 
> 4.0.33.Final, which disagrees with the version included in Flink of 
> 4.0.27.Final which is included & managed by the parent POM via dependency on 
> netty-all.
> Due to use of netty-all, the dependency management doesn't take effect on the 
> individual libraries such as netty-handler, netty-codec, etc.
> I suggest that dependency management of Netty should be added for all Netty 
> libraries individually (netty-handler, etc.) so that all Flink modules use 
> the same version, and similarly I suggest that exclusions be added to the 
> quickstart example POM for the individual Netty libraries so that fat JARs 
> don't include conflicting versions of Netty.
> It seems like this problem started when FLINK-6084 was implemented: 
> transitive dependencies of the flink-connector-cassandra were previously 
> omitted, and now that they are included we must make sure that they agree 
> with the Flink distribution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6805) Flink Cassandra connector dependency on Netty disagrees with Flink

2017-08-15 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127371#comment-16127371
 ] 

ASF GitHub Bot commented on FLINK-6805:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4545#discussion_r133215094
  
--- Diff: flink-connectors/flink-connector-cassandra/pom.xml ---
@@ -39,6 +39,7 @@ under the License.

2.2.5
3.0.0
+   4.0.27.Final
--- End diff --

is this used anywhere?


> Flink Cassandra connector dependency on Netty disagrees with Flink
> --
>
> Key: FLINK-6805
> URL: https://issues.apache.org/jira/browse/FLINK-6805
> Project: Flink
>  Issue Type: Bug
>  Components: Cassandra Connector
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Shannon Carey
>Assignee: Michael Fong
> Fix For: 1.4.0
>
>
> The Flink Cassandra connector has a dependency on Netty libraries (via 
> promotion of transitive dependencies by the Maven shade plugin) at version 
> 4.0.33.Final, which disagrees with the version included in Flink of 
> 4.0.27.Final which is included & managed by the parent POM via dependency on 
> netty-all.
> Due to use of netty-all, the dependency management doesn't take effect on the 
> individual libraries such as netty-handler, netty-codec, etc.
> I suggest that dependency management of Netty should be added for all Netty 
> libraries individually (netty-handler, etc.) so that all Flink modules use 
> the same version, and similarly I suggest that exclusions be added to the 
> quickstart example POM for the individual Netty libraries so that fat JARs 
> don't include conflicting versions of Netty.
> It seems like this problem started when FLINK-6084 was implemented: 
> transitive dependencies of the flink-connector-cassandra were previously 
> omitted, and now that they are included we must make sure that they agree 
> with the Flink distribution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6805) Flink Cassandra connector dependency on Netty disagrees with Flink

2017-08-15 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127372#comment-16127372
 ] 

ASF GitHub Bot commented on FLINK-6805:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4545#discussion_r133218223
  
--- Diff: flink-connectors/flink-connector-cassandra/pom.xml ---
@@ -83,6 +85,10 @@ under the License.

com.google.inject.**



+   
+   
io.netty
+   
org.apache.flink.shaded.netty4.io.netty
--- End diff --

you are relocating cassandra indirect netty dependency into the same 
namespace as Flink direct netty dependency, which will lead to dependency 
conflicts (because the sole way of distinguishing between class veresions, the 
package declaration, is identical).

The `shadedPattern` should be `org.apache.flink.cassandra.shaded.io.netty` 
instead.

* This is in line with the guava relocation (in the same file).
* the cassandra prefix introduces another layer of uniqueness which makes 
dependency conflicts less likely.
* I want to reserve the `org.apache.flink.shaded` namespace for 
dependencies from `flink-shaded`, to avoid accidental dependency clashes from 
being created (which would have happened had we merged the PR as is)


> Flink Cassandra connector dependency on Netty disagrees with Flink
> --
>
> Key: FLINK-6805
> URL: https://issues.apache.org/jira/browse/FLINK-6805
> Project: Flink
>  Issue Type: Bug
>  Components: Cassandra Connector
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Shannon Carey
>Assignee: Michael Fong
> Fix For: 1.4.0
>
>
> The Flink Cassandra connector has a dependency on Netty libraries (via 
> promotion of transitive dependencies by the Maven shade plugin) at version 
> 4.0.33.Final, which disagrees with the version included in Flink of 
> 4.0.27.Final which is included & managed by the parent POM via dependency on 
> netty-all.
> Due to use of netty-all, the dependency management doesn't take effect on the 
> individual libraries such as netty-handler, netty-codec, etc.
> I suggest that dependency management of Netty should be added for all Netty 
> libraries individually (netty-handler, etc.) so that all Flink modules use 
> the same version, and similarly I suggest that exclusions be added to the 
> quickstart example POM for the individual Netty libraries so that fat JARs 
> don't include conflicting versions of Netty.
> It seems like this problem started when FLINK-6084 was implemented: 
> transitive dependencies of the flink-connector-cassandra were previously 
> omitted, and now that they are included we must make sure that they agree 
> with the Flink distribution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6805) Flink Cassandra connector dependency on Netty disagrees with Flink

2017-08-15 Thread Michael Fong (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127200#comment-16127200
 ] 

Michael Fong commented on FLINK-6805:
-

Hi, [~Zentol],

Thanks for your reply. I have created a PR based on your comments above. Please 
check if this is the way it would proceed and as well as the intended netty4 
version. Thanks.



> Flink Cassandra connector dependency on Netty disagrees with Flink
> --
>
> Key: FLINK-6805
> URL: https://issues.apache.org/jira/browse/FLINK-6805
> Project: Flink
>  Issue Type: Bug
>  Components: Cassandra Connector
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Shannon Carey
>Assignee: Michael Fong
> Fix For: 1.4.0
>
>
> The Flink Cassandra connector has a dependency on Netty libraries (via 
> promotion of transitive dependencies by the Maven shade plugin) at version 
> 4.0.33.Final, which disagrees with the version included in Flink of 
> 4.0.27.Final which is included & managed by the parent POM via dependency on 
> netty-all.
> Due to use of netty-all, the dependency management doesn't take effect on the 
> individual libraries such as netty-handler, netty-codec, etc.
> I suggest that dependency management of Netty should be added for all Netty 
> libraries individually (netty-handler, etc.) so that all Flink modules use 
> the same version, and similarly I suggest that exclusions be added to the 
> quickstart example POM for the individual Netty libraries so that fat JARs 
> don't include conflicting versions of Netty.
> It seems like this problem started when FLINK-6084 was implemented: 
> transitive dependencies of the flink-connector-cassandra were previously 
> omitted, and now that they are included we must make sure that they agree 
> with the Flink distribution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6805) Flink Cassandra connector dependency on Netty disagrees with Flink

2017-08-15 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127192#comment-16127192
 ] 

ASF GitHub Bot commented on FLINK-6805:
---

GitHub user mcfongtw opened a pull request:

https://github.com/apache/flink/pull/4545

[FLINK-6805] [Cassandra-Connector] Shade indirect netty4 dep in pom.xml

## Brief change log
- To relocate various *indirect* netty4 dep of ver 4.0.33.Final to 
classpath of Flink's flavor using Maven shade plugin.

## Verifying this change
- Passed local `mvn clean verify` as well as TravisCI tests

## What is the purpose of the change
- Shade *(indirect)* netty4 dependencies to avoid class clash in the 
future. 

## Does this pull request potentially affect one of the following parts:
  - Dependencies (does it add or upgrade a dependency): (yes / no)  
**slightly related**, shade a netty4 dep.
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (yes / no) no
  - The serializers: (yes / no / don't know) no
  - The runtime per-record code paths (performance sensitive): (yes / no / 
don't know) no
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know) no

## Documentation
  - Does this pull request introduce a new feature? (yes / no) no
  - If yes, how is the feature documented? (not applicable / docs / 
JavaDocs / not documented) N.A.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcfongtw/flink FLINK-6805

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4545.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4545


commit f852eccd24b44ceece6e74bc06c24a77a9acb151
Author: Michael Fong 
Date:   2017-08-15T06:22:23Z

[FLINK-6805] [Cassandra-Connector] Shade indirect netty4 dep

To relocate various *indirect* netty4 dep of ver 4.0.33.Final to
classpath of Flink flavor using Maven shade plugin.




> Flink Cassandra connector dependency on Netty disagrees with Flink
> --
>
> Key: FLINK-6805
> URL: https://issues.apache.org/jira/browse/FLINK-6805
> Project: Flink
>  Issue Type: Bug
>  Components: Cassandra Connector
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Shannon Carey
>Assignee: Michael Fong
> Fix For: 1.4.0
>
>
> The Flink Cassandra connector has a dependency on Netty libraries (via 
> promotion of transitive dependencies by the Maven shade plugin) at version 
> 4.0.33.Final, which disagrees with the version included in Flink of 
> 4.0.27.Final which is included & managed by the parent POM via dependency on 
> netty-all.
> Due to use of netty-all, the dependency management doesn't take effect on the 
> individual libraries such as netty-handler, netty-codec, etc.
> I suggest that dependency management of Netty should be added for all Netty 
> libraries individually (netty-handler, etc.) so that all Flink modules use 
> the same version, and similarly I suggest that exclusions be added to the 
> quickstart example POM for the individual Netty libraries so that fat JARs 
> don't include conflicting versions of Netty.
> It seems like this problem started when FLINK-6084 was implemented: 
> transitive dependencies of the flink-connector-cassandra were previously 
> omitted, and now that they are included we must make sure that they agree 
> with the Flink distribution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6805) Flink Cassandra connector dependency on Netty disagrees with Flink

2017-08-14 Thread Chesnay Schepler (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125534#comment-16125534
 ] 

Chesnay Schepler commented on FLINK-6805:
-

FLINK-6982 is about shading _direct_ guava dependencies of Flink; for example 
{{flink-optimizer}} has a direct guava dependency, that (will) now use the a 
shaded version. All modules that have a direct dependency will use this one.

The cassandra module is different since it only has an indirect guava/netty 
dependency through the datastax driver. This module shades a separate version 
of guava, that isn't connected in any way to the one used by the rest of Flink. 
The same should be done with the netty dependency.

This should be very easy to implement, as you only have to modify the 
maven-shade-plugin configuration in the pom of the cassandra module, adding an 
inclusion for the netty dependencies.

> Flink Cassandra connector dependency on Netty disagrees with Flink
> --
>
> Key: FLINK-6805
> URL: https://issues.apache.org/jira/browse/FLINK-6805
> Project: Flink
>  Issue Type: Bug
>  Components: Cassandra Connector
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Shannon Carey
> Fix For: 1.4.0
>
>
> The Flink Cassandra connector has a dependency on Netty libraries (via 
> promotion of transitive dependencies by the Maven shade plugin) at version 
> 4.0.33.Final, which disagrees with the version included in Flink of 
> 4.0.27.Final which is included & managed by the parent POM via dependency on 
> netty-all.
> Due to use of netty-all, the dependency management doesn't take effect on the 
> individual libraries such as netty-handler, netty-codec, etc.
> I suggest that dependency management of Netty should be added for all Netty 
> libraries individually (netty-handler, etc.) so that all Flink modules use 
> the same version, and similarly I suggest that exclusions be added to the 
> quickstart example POM for the individual Netty libraries so that fat JARs 
> don't include conflicting versions of Netty.
> It seems like this problem started when FLINK-6084 was implemented: 
> transitive dependencies of the flink-connector-cassandra were previously 
> omitted, and now that they are included we must make sure that they agree 
> with the Flink distribution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6805) Flink Cassandra connector dependency on Netty disagrees with Flink

2017-08-09 Thread Michael Fong (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120885#comment-16120885
 ] 

Michael Fong commented on FLINK-6805:
-

Thanks for your comment, [~Zentol], 

I see the point of Flink's shaded netty and relocating original netty to Flink 
shaded netty dependency, but I don't quite get the point 2). Could you please 
share the example you mentioned for guava, and how would that prevent from 
potential conflict in future upgrade if it is not using shading approach.  
Thanks.

Regards,

Michael Fong

> Flink Cassandra connector dependency on Netty disagrees with Flink
> --
>
> Key: FLINK-6805
> URL: https://issues.apache.org/jira/browse/FLINK-6805
> Project: Flink
>  Issue Type: Bug
>  Components: Cassandra Connector
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Shannon Carey
> Fix For: 1.4.0
>
>
> The Flink Cassandra connector has a dependency on Netty libraries (via 
> promotion of transitive dependencies by the Maven shade plugin) at version 
> 4.0.33.Final, which disagrees with the version included in Flink of 
> 4.0.27.Final which is included & managed by the parent POM via dependency on 
> netty-all.
> Due to use of netty-all, the dependency management doesn't take effect on the 
> individual libraries such as netty-handler, netty-codec, etc.
> I suggest that dependency management of Netty should be added for all Netty 
> libraries individually (netty-handler, etc.) so that all Flink modules use 
> the same version, and similarly I suggest that exclusions be added to the 
> quickstart example POM for the individual Netty libraries so that fat JARs 
> don't include conflicting versions of Netty.
> It seems like this problem started when FLINK-6084 was implemented: 
> transitive dependencies of the flink-connector-cassandra were previously 
> omitted, and now that they are included we must make sure that they agree 
> with the Flink distribution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6805) Flink Cassandra connector dependency on Netty disagrees with Flink

2017-08-09 Thread Chesnay Schepler (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120018#comment-16120018
 ] 

Chesnay Schepler commented on FLINK-6805:
-

Since the cassandra connector jar doesn't contain netty Flink's dependency was 
used instead, unless a user explicitly included the datastax netty dependency.

This will no longer work since FLINK-7013 as netty is now shaded away.

There are 2 ways to fix this:
# relocate netty references in the datastax dependency to re-use the shaded 
netty dependency
# include the datastax netty dependency in the cassandra jar

I'm leaning towards 2). We are doing the same for guava, and is less likely to 
blow up in the future when either Flink or datastax move to a different netty 
version.

> Flink Cassandra connector dependency on Netty disagrees with Flink
> --
>
> Key: FLINK-6805
> URL: https://issues.apache.org/jira/browse/FLINK-6805
> Project: Flink
>  Issue Type: Bug
>  Components: Cassandra Connector
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Shannon Carey
>
> The Flink Cassandra connector has a dependency on Netty libraries (via 
> promotion of transitive dependencies by the Maven shade plugin) at version 
> 4.0.33.Final, which disagrees with the version included in Flink of 
> 4.0.27.Final which is included & managed by the parent POM via dependency on 
> netty-all.
> Due to use of netty-all, the dependency management doesn't take effect on the 
> individual libraries such as netty-handler, netty-codec, etc.
> I suggest that dependency management of Netty should be added for all Netty 
> libraries individually (netty-handler, etc.) so that all Flink modules use 
> the same version, and similarly I suggest that exclusions be added to the 
> quickstart example POM for the individual Netty libraries so that fat JARs 
> don't include conflicting versions of Netty.
> It seems like this problem started when FLINK-6084 was implemented: 
> transitive dependencies of the flink-connector-cassandra were previously 
> omitted, and now that they are included we must make sure that they agree 
> with the Flink distribution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6805) Flink Cassandra connector dependency on Netty disagrees with Flink

2017-08-09 Thread Michael Fong (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119971#comment-16119971
 ] 

Michael Fong commented on FLINK-6805:
-

All other flink libraries depends on 4.0.27, except flink-connect-cassandra 
seems to have a newer Netty4 dependency of version 4.0.33 using default scope 
'compile'.

Resolved dep tree:
[INFO] org.apache.flink:flink-connector-cassandra_2.11:jar:1.4-SNAPSHOT
[INFO] +- 
{color:red}com.datastax.cassandra:cassandra-driver-core:jar:3.0.{color}0:compile
[INFO] |  \- io.netty:netty-handler:jar:4.0.33.Final:compile
[INFO] | +- io.netty:netty-buffer:jar:4.0.33.Final:compile
[INFO] | |  \- io.netty:netty-common:jar:4.0.33.Final:compile
[INFO] | +- io.netty:netty-transport:jar:4.0.33.Final:compile
[INFO] | |  \- (io.netty:netty-buffer:jar:4.0.33.Final:compile - omitted 
for duplicate)
[INFO] | \- io.netty:netty-codec:jar:4.0.33.Final:compile
[INFO] |\- (io.netty:netty-transport:jar:4.0.33.Final:compile - omitted 
for duplicate)
[INFO] +- org.apache.flink:flink-runtime_2.11:test-jar:tests:1.4-SNAPSHOT:test
[INFO] |  +- com.data-artisans:flakka-remote_2.11:jar:2.3-custom:test
[INFO] |  |  \- io.netty:netty:jar:3.8.0.Final:test
[INFO] |  \- org.apache.zookeeper:zookeeper:jar:3.4.10:test
[INFO] | \- (io.netty:netty:jar:3.10.5.Final:test - omitted for conflict 
with 3.8.0.Final)
[INFO] \- org.apache.cassandra:cassandra-all:jar:2.2.5:test
[INFO]\- io.netty:netty-all:jar:4.0.23.Final:test

However, I am not sure which version would be picked up at runtime, since Flink 
seems to wrap all classes (libraries included) in a fat jar at runtime. In 
addition, there is the FLINK-7013 that intends to organize all classes imported 
from netty. 

Hi, [~Zentol], could you please share some insight to this tickets? I would 
love to help improve on this if there is any need to update maven dependencies. 

Regards,

Michael Fong

> Flink Cassandra connector dependency on Netty disagrees with Flink
> --
>
> Key: FLINK-6805
> URL: https://issues.apache.org/jira/browse/FLINK-6805
> Project: Flink
>  Issue Type: Bug
>  Components: Cassandra Connector
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Shannon Carey
>
> The Flink Cassandra connector has a dependency on Netty libraries (via 
> promotion of transitive dependencies by the Maven shade plugin) at version 
> 4.0.33.Final, which disagrees with the version included in Flink of 
> 4.0.27.Final which is included & managed by the parent POM via dependency on 
> netty-all.
> Due to use of netty-all, the dependency management doesn't take effect on the 
> individual libraries such as netty-handler, netty-codec, etc.
> I suggest that dependency management of Netty should be added for all Netty 
> libraries individually (netty-handler, etc.) so that all Flink modules use 
> the same version, and similarly I suggest that exclusions be added to the 
> quickstart example POM for the individual Netty libraries so that fat JARs 
> don't include conflicting versions of Netty.
> It seems like this problem started when FLINK-6084 was implemented: 
> transitive dependencies of the flink-connector-cassandra were previously 
> omitted, and now that they are included we must make sure that they agree 
> with the Flink distribution.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-6805) Flink Cassandra connector dependency on Netty disagrees with Flink

2017-06-01 Thread Shannon Carey (JIRA) 

[ 
https://issues.apache.org/jira/browse/FLINK-6805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033571#comment-16033571
 ] 

Shannon Carey commented on FLINK-6805:
--

A better solution may be to add exclusions for the Netty libraries in the 
flink-connector-cassandra POM, then the other changes wouldn't necessarily be 
required (although they might still be desirable).

> Flink Cassandra connector dependency on Netty disagrees with Flink
> --
>
> Key: FLINK-6805
> URL: https://issues.apache.org/jira/browse/FLINK-6805
> Project: Flink
>  Issue Type: Bug
>  Components: Cassandra Connector
>Affects Versions: 1.3.0, 1.2.1
>Reporter: Shannon Carey
>
> The Flink Cassandra connector has a dependency on Netty libraries (via 
> promotion of transitive dependencies by the Maven shade plugin) at version 
> 4.0.33.Final, which disagrees with the version included in Flink of 
> 4.0.27.Final which is included & managed by the parent POM via dependency on 
> netty-all.
> Due to use of netty-all, the dependency management doesn't take effect on the 
> individual libraries such as netty-handler, netty-codec, etc.
> I suggest that dependency management of Netty should be added for all Netty 
> libraries individually (netty-handler, etc.) so that all Flink modules use 
> the same version, and similarly I suggest that exclusions be added to the 
> quickstart example POM for the individual Netty libraries so that fat JARs 
> don't include conflicting versions of Netty.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)