[jira] [Commented] (FLINK-2584) ASM dependency is not shaded away
[ https://issues.apache.org/jira/browse/FLINK-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14726934#comment-14726934 ] ASF GitHub Bot commented on FLINK-2584: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/1076 > ASM dependency is not shaded away > - > > Key: FLINK-2584 > URL: https://issues.apache.org/jira/browse/FLINK-2584 > Project: Flink > Issue Type: Bug > Components: Core >Affects Versions: 0.9, master >Reporter: Ufuk Celebi >Assignee: Stephan Ewen > Fix For: 0.10, 0.9.1 > > > ASM is not correctly shaded away. If you build the quick start against the > snapshot version, you will see the following dependencies. Robert is fixing > this. > {code} > [INFO] +- org.apache.flink:flink-java:jar:0.9.1:compile > [INFO] | +- org.apache.flink:flink-core:jar:0.9.1:compile > [INFO] | | \- commons-collections:commons-collections:jar:3.2.1:compile > [INFO] | +- org.apache.flink:flink-shaded-include-yarn:jar:0.9.1:compile > [INFO] | +- org.apache.avro:avro:jar:1.7.6:compile > [INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile > [INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile > [INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile > [INFO] | | +- org.xerial.snappy:snappy-java:jar:1.0.5:compile > [INFO] | | \- org.apache.commons:commons-compress:jar:1.4.1:compile > [INFO] | | \- org.tukaani:xz:jar:1.0:compile > [INFO] | +- com.esotericsoftware.kryo:kryo:jar:2.24.0:compile > [INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile > [INFO] | | \- org.objenesis:objenesis:jar:2.1:compile > [INFO] | +- com.twitter:chill_2.10:jar:0.5.2:compile > [INFO] | | +- org.scala-lang:scala-library:jar:2.10.4:compile > [INFO] | | \- com.twitter:chill-java:jar:0.5.2:compile > [INFO] | +- com.twitter:chill-avro_2.10:jar:0.5.2:compile > [INFO] | | +- com.twitter:chill-bijection_2.10:jar:0.5.2:compile > [INFO] | | | \- com.twitter:bijection-core_2.10:jar:0.7.2:compile > [INFO] | | \- com.twitter:bijection-avro_2.10:jar:0.7.2:compile > [INFO] | +- de.javakaffee:kryo-serializers:jar:0.36:compile > [INFO] | | +- com.esotericsoftware:kryo:jar:3.0.3:compile > [INFO] | | | +- com.esotericsoftware:reflectasm:jar:1.10.1:compile > [INFO] | | | | \- org.ow2.asm:asm:jar:5.0.3:compile > [INFO] | | | \- com.esotericsoftware:minlog:jar:1.3.0:compile > [INFO] | | \- com.google.protobuf:protobuf-java:jar:2.6.1:compile > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2584) ASM dependency is not shaded away
[ https://issues.apache.org/jira/browse/FLINK-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14723310#comment-14723310 ] ASF GitHub Bot commented on FLINK-2584: --- Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/1076#issuecomment-136339437 Make sure it does not break with @tillrohrmann's recent merge ;) > ASM dependency is not shaded away > - > > Key: FLINK-2584 > URL: https://issues.apache.org/jira/browse/FLINK-2584 > Project: Flink > Issue Type: Bug > Components: Core >Affects Versions: 0.9, master >Reporter: Ufuk Celebi >Assignee: Stephan Ewen > Fix For: 0.10, 0.9.1 > > > ASM is not correctly shaded away. If you build the quick start against the > snapshot version, you will see the following dependencies. Robert is fixing > this. > {code} > [INFO] +- org.apache.flink:flink-java:jar:0.9.1:compile > [INFO] | +- org.apache.flink:flink-core:jar:0.9.1:compile > [INFO] | | \- commons-collections:commons-collections:jar:3.2.1:compile > [INFO] | +- org.apache.flink:flink-shaded-include-yarn:jar:0.9.1:compile > [INFO] | +- org.apache.avro:avro:jar:1.7.6:compile > [INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile > [INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile > [INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile > [INFO] | | +- org.xerial.snappy:snappy-java:jar:1.0.5:compile > [INFO] | | \- org.apache.commons:commons-compress:jar:1.4.1:compile > [INFO] | | \- org.tukaani:xz:jar:1.0:compile > [INFO] | +- com.esotericsoftware.kryo:kryo:jar:2.24.0:compile > [INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile > [INFO] | | \- org.objenesis:objenesis:jar:2.1:compile > [INFO] | +- com.twitter:chill_2.10:jar:0.5.2:compile > [INFO] | | +- org.scala-lang:scala-library:jar:2.10.4:compile > [INFO] | | \- com.twitter:chill-java:jar:0.5.2:compile > [INFO] | +- com.twitter:chill-avro_2.10:jar:0.5.2:compile > [INFO] | | +- com.twitter:chill-bijection_2.10:jar:0.5.2:compile > [INFO] | | | \- com.twitter:bijection-core_2.10:jar:0.7.2:compile > [INFO] | | \- com.twitter:bijection-avro_2.10:jar:0.7.2:compile > [INFO] | +- de.javakaffee:kryo-serializers:jar:0.36:compile > [INFO] | | +- com.esotericsoftware:kryo:jar:3.0.3:compile > [INFO] | | | +- com.esotericsoftware:reflectasm:jar:1.10.1:compile > [INFO] | | | | \- org.ow2.asm:asm:jar:5.0.3:compile > [INFO] | | | \- com.esotericsoftware:minlog:jar:1.3.0:compile > [INFO] | | \- com.google.protobuf:protobuf-java:jar:2.6.1:compile > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2584) ASM dependency is not shaded away
[ https://issues.apache.org/jira/browse/FLINK-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14723305#comment-14723305 ] ASF GitHub Bot commented on FLINK-2584: --- Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1076#issuecomment-136336944 I'll merge this now into `master` and `0.10.0-milestone1`. In the long run, we can hopefully convince curator to offer a shaded version in addition to the unshaded version. > ASM dependency is not shaded away > - > > Key: FLINK-2584 > URL: https://issues.apache.org/jira/browse/FLINK-2584 > Project: Flink > Issue Type: Bug > Components: Core >Affects Versions: 0.9, master >Reporter: Ufuk Celebi >Assignee: Stephan Ewen > Fix For: 0.10, 0.9.1 > > > ASM is not correctly shaded away. If you build the quick start against the > snapshot version, you will see the following dependencies. Robert is fixing > this. > {code} > [INFO] +- org.apache.flink:flink-java:jar:0.9.1:compile > [INFO] | +- org.apache.flink:flink-core:jar:0.9.1:compile > [INFO] | | \- commons-collections:commons-collections:jar:3.2.1:compile > [INFO] | +- org.apache.flink:flink-shaded-include-yarn:jar:0.9.1:compile > [INFO] | +- org.apache.avro:avro:jar:1.7.6:compile > [INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile > [INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile > [INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile > [INFO] | | +- org.xerial.snappy:snappy-java:jar:1.0.5:compile > [INFO] | | \- org.apache.commons:commons-compress:jar:1.4.1:compile > [INFO] | | \- org.tukaani:xz:jar:1.0:compile > [INFO] | +- com.esotericsoftware.kryo:kryo:jar:2.24.0:compile > [INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile > [INFO] | | \- org.objenesis:objenesis:jar:2.1:compile > [INFO] | +- com.twitter:chill_2.10:jar:0.5.2:compile > [INFO] | | +- org.scala-lang:scala-library:jar:2.10.4:compile > [INFO] | | \- com.twitter:chill-java:jar:0.5.2:compile > [INFO] | +- com.twitter:chill-avro_2.10:jar:0.5.2:compile > [INFO] | | +- com.twitter:chill-bijection_2.10:jar:0.5.2:compile > [INFO] | | | \- com.twitter:bijection-core_2.10:jar:0.7.2:compile > [INFO] | | \- com.twitter:bijection-avro_2.10:jar:0.7.2:compile > [INFO] | +- de.javakaffee:kryo-serializers:jar:0.36:compile > [INFO] | | +- com.esotericsoftware:kryo:jar:3.0.3:compile > [INFO] | | | +- com.esotericsoftware:reflectasm:jar:1.10.1:compile > [INFO] | | | | \- org.ow2.asm:asm:jar:5.0.3:compile > [INFO] | | | \- com.esotericsoftware:minlog:jar:1.3.0:compile > [INFO] | | \- com.google.protobuf:protobuf-java:jar:2.6.1:compile > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2584) ASM dependency is not shaded away
[ https://issues.apache.org/jira/browse/FLINK-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14721488#comment-14721488 ] ASF GitHub Bot commented on FLINK-2584: --- GitHub user rmetzger opened a pull request: https://github.com/apache/flink/pull/1076 [FLINK-2584] Check for unshaded classes in fat jar and shade curator This PR is an addition for FLINK-2584, removing the transitive guava dependencies from the fat jar introduced by Apache Curator. Its also adding a check ensuring that shaded classes (guava, asm) are not showing up in the fat jar. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rmetzger/flink flink2584 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/1076.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1076 commit 7ae0c9d956ffcdb77d55edc70f1588dc507a9c39 Author: Robert Metzger rmetz...@apache.org Date: 2015-08-27T16:13:08Z [FLINK-2584] Check for unshaded classes in fat jar and shade curator ASM dependency is not shaded away - Key: FLINK-2584 URL: https://issues.apache.org/jira/browse/FLINK-2584 Project: Flink Issue Type: Bug Components: Core Affects Versions: 0.9, master Reporter: Ufuk Celebi Assignee: Stephan Ewen Fix For: 0.10, 0.9.1 ASM is not correctly shaded away. If you build the quick start against the snapshot version, you will see the following dependencies. Robert is fixing this. {code} [INFO] +- org.apache.flink:flink-java:jar:0.9.1:compile [INFO] | +- org.apache.flink:flink-core:jar:0.9.1:compile [INFO] | | \- commons-collections:commons-collections:jar:3.2.1:compile [INFO] | +- org.apache.flink:flink-shaded-include-yarn:jar:0.9.1:compile [INFO] | +- org.apache.avro:avro:jar:1.7.6:compile [INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile [INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile [INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile [INFO] | | +- org.xerial.snappy:snappy-java:jar:1.0.5:compile [INFO] | | \- org.apache.commons:commons-compress:jar:1.4.1:compile [INFO] | | \- org.tukaani:xz:jar:1.0:compile [INFO] | +- com.esotericsoftware.kryo:kryo:jar:2.24.0:compile [INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile [INFO] | | \- org.objenesis:objenesis:jar:2.1:compile [INFO] | +- com.twitter:chill_2.10:jar:0.5.2:compile [INFO] | | +- org.scala-lang:scala-library:jar:2.10.4:compile [INFO] | | \- com.twitter:chill-java:jar:0.5.2:compile [INFO] | +- com.twitter:chill-avro_2.10:jar:0.5.2:compile [INFO] | | +- com.twitter:chill-bijection_2.10:jar:0.5.2:compile [INFO] | | | \- com.twitter:bijection-core_2.10:jar:0.7.2:compile [INFO] | | \- com.twitter:bijection-avro_2.10:jar:0.7.2:compile [INFO] | +- de.javakaffee:kryo-serializers:jar:0.36:compile [INFO] | | +- com.esotericsoftware:kryo:jar:3.0.3:compile [INFO] | | | +- com.esotericsoftware:reflectasm:jar:1.10.1:compile [INFO] | | | | \- org.ow2.asm:asm:jar:5.0.3:compile [INFO] | | | \- com.esotericsoftware:minlog:jar:1.3.0:compile [INFO] | | \- com.google.protobuf:protobuf-java:jar:2.6.1:compile {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2584) ASM dependency is not shaded away
[ https://issues.apache.org/jira/browse/FLINK-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14721503#comment-14721503 ] ASF GitHub Bot commented on FLINK-2584: --- Github user uce commented on the pull request: https://github.com/apache/flink/pull/1076#issuecomment-136131254 Nice work :) This needs to go into the milestone branch as well. What do you mean with We can easily integrate curator's netty into the jar file.? ASM dependency is not shaded away - Key: FLINK-2584 URL: https://issues.apache.org/jira/browse/FLINK-2584 Project: Flink Issue Type: Bug Components: Core Affects Versions: 0.9, master Reporter: Ufuk Celebi Assignee: Stephan Ewen Fix For: 0.10, 0.9.1 ASM is not correctly shaded away. If you build the quick start against the snapshot version, you will see the following dependencies. Robert is fixing this. {code} [INFO] +- org.apache.flink:flink-java:jar:0.9.1:compile [INFO] | +- org.apache.flink:flink-core:jar:0.9.1:compile [INFO] | | \- commons-collections:commons-collections:jar:3.2.1:compile [INFO] | +- org.apache.flink:flink-shaded-include-yarn:jar:0.9.1:compile [INFO] | +- org.apache.avro:avro:jar:1.7.6:compile [INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile [INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile [INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile [INFO] | | +- org.xerial.snappy:snappy-java:jar:1.0.5:compile [INFO] | | \- org.apache.commons:commons-compress:jar:1.4.1:compile [INFO] | | \- org.tukaani:xz:jar:1.0:compile [INFO] | +- com.esotericsoftware.kryo:kryo:jar:2.24.0:compile [INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile [INFO] | | \- org.objenesis:objenesis:jar:2.1:compile [INFO] | +- com.twitter:chill_2.10:jar:0.5.2:compile [INFO] | | +- org.scala-lang:scala-library:jar:2.10.4:compile [INFO] | | \- com.twitter:chill-java:jar:0.5.2:compile [INFO] | +- com.twitter:chill-avro_2.10:jar:0.5.2:compile [INFO] | | +- com.twitter:chill-bijection_2.10:jar:0.5.2:compile [INFO] | | | \- com.twitter:bijection-core_2.10:jar:0.7.2:compile [INFO] | | \- com.twitter:bijection-avro_2.10:jar:0.7.2:compile [INFO] | +- de.javakaffee:kryo-serializers:jar:0.36:compile [INFO] | | +- com.esotericsoftware:kryo:jar:3.0.3:compile [INFO] | | | +- com.esotericsoftware:reflectasm:jar:1.10.1:compile [INFO] | | | | \- org.ow2.asm:asm:jar:5.0.3:compile [INFO] | | | \- com.esotericsoftware:minlog:jar:1.3.0:compile [INFO] | | \- com.google.protobuf:protobuf-java:jar:2.6.1:compile {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2584) ASM dependency is not shaded away
[ https://issues.apache.org/jira/browse/FLINK-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14721504#comment-14721504 ] ASF GitHub Bot commented on FLINK-2584: --- Github user rmetzger commented on the pull request: https://github.com/apache/flink/pull/1076#issuecomment-136131482 I saw that curator has a netty dependency. If we are running into netty conflicts because of curator's netty dependency, it's very easy to just shade curator's netty into curator. My change is creating a new apache curator jar for us, where guava is located in `org.apache.curator.shaded.com.google`. We can do exactly the same for netty if needed. ASM dependency is not shaded away - Key: FLINK-2584 URL: https://issues.apache.org/jira/browse/FLINK-2584 Project: Flink Issue Type: Bug Components: Core Affects Versions: 0.9, master Reporter: Ufuk Celebi Assignee: Stephan Ewen Fix For: 0.10, 0.9.1 ASM is not correctly shaded away. If you build the quick start against the snapshot version, you will see the following dependencies. Robert is fixing this. {code} [INFO] +- org.apache.flink:flink-java:jar:0.9.1:compile [INFO] | +- org.apache.flink:flink-core:jar:0.9.1:compile [INFO] | | \- commons-collections:commons-collections:jar:3.2.1:compile [INFO] | +- org.apache.flink:flink-shaded-include-yarn:jar:0.9.1:compile [INFO] | +- org.apache.avro:avro:jar:1.7.6:compile [INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile [INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile [INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile [INFO] | | +- org.xerial.snappy:snappy-java:jar:1.0.5:compile [INFO] | | \- org.apache.commons:commons-compress:jar:1.4.1:compile [INFO] | | \- org.tukaani:xz:jar:1.0:compile [INFO] | +- com.esotericsoftware.kryo:kryo:jar:2.24.0:compile [INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile [INFO] | | \- org.objenesis:objenesis:jar:2.1:compile [INFO] | +- com.twitter:chill_2.10:jar:0.5.2:compile [INFO] | | +- org.scala-lang:scala-library:jar:2.10.4:compile [INFO] | | \- com.twitter:chill-java:jar:0.5.2:compile [INFO] | +- com.twitter:chill-avro_2.10:jar:0.5.2:compile [INFO] | | +- com.twitter:chill-bijection_2.10:jar:0.5.2:compile [INFO] | | | \- com.twitter:bijection-core_2.10:jar:0.7.2:compile [INFO] | | \- com.twitter:bijection-avro_2.10:jar:0.7.2:compile [INFO] | +- de.javakaffee:kryo-serializers:jar:0.36:compile [INFO] | | +- com.esotericsoftware:kryo:jar:3.0.3:compile [INFO] | | | +- com.esotericsoftware:reflectasm:jar:1.10.1:compile [INFO] | | | | \- org.ow2.asm:asm:jar:5.0.3:compile [INFO] | | | \- com.esotericsoftware:minlog:jar:1.3.0:compile [INFO] | | \- com.google.protobuf:protobuf-java:jar:2.6.1:compile {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-2584) ASM dependency is not shaded away
[ https://issues.apache.org/jira/browse/FLINK-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14716872#comment-14716872 ] Robert Metzger commented on FLINK-2584: --- The problem was that we upgraded the dependency {{de.javakaffee:kryo-serializers}} from 0.27 to 0.36. I will add a test into our travis build scripts which ensures that classes we expect to be shaded are not in the fat jar. ASM dependency is not shaded away - Key: FLINK-2584 URL: https://issues.apache.org/jira/browse/FLINK-2584 Project: Flink Issue Type: Bug Components: Core Affects Versions: 0.9, master Reporter: Ufuk Celebi Assignee: Robert Metzger ASM is not correctly shaded away. If you build the quick start against the snapshot version, you will see the following dependencies. Robert is fixing this. {code} [INFO] +- org.apache.flink:flink-java:jar:0.9.1:compile [INFO] | +- org.apache.flink:flink-core:jar:0.9.1:compile [INFO] | | \- commons-collections:commons-collections:jar:3.2.1:compile [INFO] | +- org.apache.flink:flink-shaded-include-yarn:jar:0.9.1:compile [INFO] | +- org.apache.avro:avro:jar:1.7.6:compile [INFO] | | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile [INFO] | | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile [INFO] | | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile [INFO] | | +- org.xerial.snappy:snappy-java:jar:1.0.5:compile [INFO] | | \- org.apache.commons:commons-compress:jar:1.4.1:compile [INFO] | | \- org.tukaani:xz:jar:1.0:compile [INFO] | +- com.esotericsoftware.kryo:kryo:jar:2.24.0:compile [INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile [INFO] | | \- org.objenesis:objenesis:jar:2.1:compile [INFO] | +- com.twitter:chill_2.10:jar:0.5.2:compile [INFO] | | +- org.scala-lang:scala-library:jar:2.10.4:compile [INFO] | | \- com.twitter:chill-java:jar:0.5.2:compile [INFO] | +- com.twitter:chill-avro_2.10:jar:0.5.2:compile [INFO] | | +- com.twitter:chill-bijection_2.10:jar:0.5.2:compile [INFO] | | | \- com.twitter:bijection-core_2.10:jar:0.7.2:compile [INFO] | | \- com.twitter:bijection-avro_2.10:jar:0.7.2:compile [INFO] | +- de.javakaffee:kryo-serializers:jar:0.36:compile [INFO] | | +- com.esotericsoftware:kryo:jar:3.0.3:compile [INFO] | | | +- com.esotericsoftware:reflectasm:jar:1.10.1:compile [INFO] | | | | \- org.ow2.asm:asm:jar:5.0.3:compile [INFO] | | | \- com.esotericsoftware:minlog:jar:1.3.0:compile [INFO] | | \- com.google.protobuf:protobuf-java:jar:2.6.1:compile {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)