[
https://issues.apache.org/jira/browse/FLINK-3247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110312#comment-15110312
]
Robert Metzger commented on FLINK-3247:
---------------------------------------
I'm not sure, even with Maven 3.0.4, maven will include curator into the fat
jar .. so I can not reproduce the issue.
{code}
../apache-maven-3.0.4/bin/mvn clean install
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Flink Quickstart Job 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ debug-1.0-2.11 ---
[INFO] Deleting /home/robert/flink-workdir/debug-1.0-2.11/target
[INFO]
[INFO] --- build-helper-maven-plugin:1.7:add-source (add-source) @
debug-1.0-2.11 ---
[INFO] Source directory:
/home/robert/flink-workdir/debug-1.0-2.11/src/main/scala added.
[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @
debug-1.0-2.11 ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ debug-1.0-2.11
---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- scala-maven-plugin:3.1.4:compile (default) @ debug-1.0-2.11 ---
[WARNING] Expected all dependencies to require Scala version: 2.10.4
[WARNING] org.scala-lang:scala-reflect:2.10.4 requires scala version: 2.10.4
[WARNING] org.apache.flink:flink-scala:1.0-SNAPSHOT requires scala version:
2.10.4
[WARNING] org.apache.flink:flink-scala:1.0-SNAPSHOT requires scala version:
2.10.4
[WARNING] org.scala-lang:scala-compiler:2.10.4 requires scala version: 2.10.4
[WARNING] org.scalamacros:quasiquotes_2.10:2.0.1 requires scala version: 2.10.4
[WARNING] org.apache.flink:flink-streaming-scala:1.0-SNAPSHOT requires scala
version: 2.10.4
[WARNING] org.apache.flink:flink-streaming-scala:1.0-SNAPSHOT requires scala
version: 2.10.4
[WARNING] org.apache.flink:flink-runtime:1.0-SNAPSHOT requires scala version:
2.10.4
[WARNING] com.typesafe.akka:akka-actor_2.10:2.3.7 requires scala version:
2.10.4
[WARNING] com.typesafe.akka:akka-remote_2.10:2.3.7 requires scala version:
2.10.4
[WARNING] com.typesafe.akka:akka-slf4j_2.10:2.3.7 requires scala version:
2.10.4
[WARNING] org.clapper:grizzled-slf4j_2.10:1.0.2 requires scala version: 2.10.3
[WARNING] Multiple versions of scala libraries detected!
[INFO] /home/robert/flink-workdir/debug-1.0-2.11/src/main/scala:-1: info:
compiling
[INFO] Compiling 4 source files to
/home/robert/flink-workdir/debug-1.0-2.11/target/classes at 1453367291871
[WARNING] warning: there were 1 feature warning(s); re-run with -feature for
details
[WARNING] one warning found
[INFO] prepare-compile in 0 s
[INFO] compile in 3 s
[INFO]
[INFO] --- build-helper-maven-plugin:1.7:add-test-source (add-test-source) @
debug-1.0-2.11 ---
[INFO] Test Source directory:
/home/robert/flink-workdir/debug-1.0-2.11/src/test/scala added.
[INFO]
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @
debug-1.0-2.11 ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/home/robert/flink-workdir/debug-1.0-2.11/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @
debug-1.0-2.11 ---
[INFO] No sources to compile
[INFO]
[INFO] --- scala-maven-plugin:3.1.4:testCompile (default) @ debug-1.0-2.11 ---
[WARNING] Expected all dependencies to require Scala version: 2.10.4
[WARNING] org.scala-lang:scala-reflect:2.10.4 requires scala version: 2.10.4
[WARNING] org.apache.flink:flink-scala:1.0-SNAPSHOT requires scala version:
2.10.4
[WARNING] org.apache.flink:flink-scala:1.0-SNAPSHOT requires scala version:
2.10.4
[WARNING] org.scala-lang:scala-compiler:2.10.4 requires scala version: 2.10.4
[WARNING] org.scalamacros:quasiquotes_2.10:2.0.1 requires scala version: 2.10.4
[WARNING] org.apache.flink:flink-streaming-scala:1.0-SNAPSHOT requires scala
version: 2.10.4
[WARNING] org.apache.flink:flink-streaming-scala:1.0-SNAPSHOT requires scala
version: 2.10.4
[WARNING] org.apache.flink:flink-runtime:1.0-SNAPSHOT requires scala version:
2.10.4
[WARNING] com.typesafe.akka:akka-actor_2.10:2.3.7 requires scala version:
2.10.4
[WARNING] com.typesafe.akka:akka-remote_2.10:2.3.7 requires scala version:
2.10.4
[WARNING] com.typesafe.akka:akka-slf4j_2.10:2.3.7 requires scala version:
2.10.4
[WARNING] org.clapper:grizzled-slf4j_2.10:1.0.2 requires scala version: 2.10.3
[WARNING] Multiple versions of scala libraries detected!
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.10:test (default-test) @ debug-1.0-2.11 ---
[INFO] No tests to run.
[INFO] Surefire report directory:
/home/robert/flink-workdir/debug-1.0-2.11/target/surefire-reports
Downloading:
http://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit3/2.10/surefire-junit3-2.10.pom
Downloaded:
http://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit3/2.10/surefire-junit3-2.10.pom
(2 KB at 10.7 KB/sec)
Downloading:
http://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/2.10/surefire-providers-2.10.pom
Downloaded:
http://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/2.10/surefire-providers-2.10.pom
(3 KB at 42.6 KB/sec)
Downloading:
http://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit3/2.10/surefire-junit3-2.10.jar
Downloaded:
http://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit3/2.10/surefire-junit3-2.10.jar
(26 KB at 349.7 KB/sec)
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Results :
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] --- maven-jar-plugin:2.3.2:jar (default-jar) @ debug-1.0-2.11 ---
[INFO] Building jar:
/home/robert/flink-workdir/debug-1.0-2.11/target/debug-1.0-2.11-1.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-shade-plugin:2.4.1:shade (default) @ debug-1.0-2.11 ---
[INFO] Including org.apache.flink:flink-scala:jar:1.0-SNAPSHOT in the shaded
jar.
[INFO] Including org.apache.flink:flink-core:jar:1.0-SNAPSHOT in the shaded jar.
[INFO] Including org.apache.flink:flink-annotations:jar:1.0-SNAPSHOT in the
shaded jar.
[INFO] Including org.apache.flink:flink-shaded-hadoop2:jar:1.0-SNAPSHOT in the
shaded jar.
[INFO] Including xmlenc:xmlenc:jar:0.52 in the shaded jar.
[INFO] Including commons-httpclient:commons-httpclient:jar:3.1 in the shaded
jar.
[INFO] Excluding commons-codec:commons-codec:jar:1.4 from the shaded jar.
[INFO] Excluding commons-io:commons-io:jar:2.4 from the shaded jar.
[INFO] Including commons-net:commons-net:jar:3.1 in the shaded jar.
[INFO] Including javax.servlet:servlet-api:jar:2.5 in the shaded jar.
[INFO] Including com.sun.jersey:jersey-core:jar:1.9 in the shaded jar.
[INFO] Including commons-el:commons-el:jar:1.0 in the shaded jar.
[INFO] Excluding commons-logging:commons-logging:jar:1.1.3 from the shaded jar.
[INFO] Including net.java.dev.jets3t:jets3t:jar:0.9.0 in the shaded jar.
[INFO] Excluding org.apache.httpcomponents:httpclient:jar:4.2.6 from the shaded
jar.
[INFO] Excluding org.apache.httpcomponents:httpcore:jar:4.2.5 from the shaded
jar.
[INFO] Including com.jamesmurty.utils:java-xmlbuilder:jar:0.4 in the shaded jar.
[INFO] Excluding commons-lang:commons-lang:jar:2.6 from the shaded jar.
[INFO] Including commons-configuration:commons-configuration:jar:1.6 in the
shaded jar.
[INFO] Including commons-digester:commons-digester:jar:1.8 in the shaded jar.
[INFO] Including commons-beanutils:commons-beanutils:jar:1.7.0 in the shaded
jar.
[INFO] Including commons-beanutils:commons-beanutils-core:jar:1.8.0 in the
shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-core-asl:jar:1.8.8 from the
shaded jar.
[INFO] Excluding org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8 from the
shaded jar.
[INFO] Excluding com.thoughtworks.paranamer:paranamer:jar:2.3 from the shaded
jar.
[INFO] Excluding org.xerial.snappy:snappy-java:jar:1.0.5 from the shaded jar.
[INFO] Including com.jcraft:jsch:jar:0.1.42 in the shaded jar.
[INFO] Excluding org.apache.commons:commons-compress:jar:1.4.1 from the shaded
jar.
[INFO] Excluding org.tukaani:xz:jar:1.0 from the shaded jar.
[INFO] Including commons-daemon:commons-daemon:jar:1.0.13 in the shaded jar.
[INFO] Including javax.xml.bind:jaxb-api:jar:2.2.2 in the shaded jar.
[INFO] Including javax.xml.stream:stax-api:jar:1.0-2 in the shaded jar.
[INFO] Including javax.activation:activation:jar:1.1 in the shaded jar.
[INFO] Including com.google.inject:guice:jar:3.0 in the shaded jar.
[INFO] Including javax.inject:javax.inject:jar:1 in the shaded jar.
[INFO] Including aopalliance:aopalliance:jar:1.0 in the shaded jar.
[INFO] Excluding commons-collections:commons-collections:jar:3.2.2 from the
shaded jar.
[INFO] Excluding com.esotericsoftware.kryo:kryo:jar:2.24.0 from the shaded jar.
[INFO] Excluding com.esotericsoftware.minlog:minlog:jar:1.2 from the shaded jar.
[INFO] Excluding org.objenesis:objenesis:jar:2.1 from the shaded jar.
[INFO] Including org.apache.flink:flink-java:jar:1.0-SNAPSHOT in the shaded jar.
[INFO] Excluding org.apache.avro:avro:jar:1.7.6 from the shaded jar.
[INFO] Excluding com.twitter:chill-java:jar:0.5.2 from the shaded jar.
[INFO] Excluding de.javakaffee:kryo-serializers:jar:0.27 from the shaded jar.
[INFO] Excluding joda-time:joda-time:jar:2.5 from the shaded jar.
[INFO] Including org.joda:joda-convert:jar:1.7 in the shaded jar.
[INFO] Including org.apache.commons:commons-math3:jar:3.5 in the shaded jar.
[INFO] Including org.apache.flink:flink-optimizer:jar:1.0-SNAPSHOT in the
shaded jar.
[INFO] Excluding org.scala-lang:scala-reflect:jar:2.10.4 from the shaded jar.
[INFO] Excluding org.scala-lang:scala-library:jar:2.10.4 from the shaded jar.
[INFO] Excluding org.scala-lang:scala-compiler:jar:2.10.4 from the shaded jar.
[INFO] Including org.scalamacros:quasiquotes_2.10:jar:2.0.1 in the shaded jar.
[INFO] Excluding org.apache.commons:commons-lang3:jar:3.3.2 from the shaded jar.
[INFO] Excluding org.slf4j:slf4j-api:jar:1.7.7 from the shaded jar.
[INFO] Excluding org.slf4j:slf4j-log4j12:jar:1.7.7 from the shaded jar.
[INFO] Excluding log4j:log4j:jar:1.2.17 from the shaded jar.
[INFO] Including org.apache.flink:flink-streaming-scala:jar:1.0-SNAPSHOT in the
shaded jar.
[INFO] Including org.apache.flink:flink-streaming-java:jar:1.0-SNAPSHOT in the
shaded jar.
[INFO] Excluding org.apache.commons:commons-math:jar:2.2 from the shaded jar.
[INFO] Excluding org.apache.sling:org.apache.sling.commons.json:jar:2.0.6 from
the shaded jar.
[INFO] Including org.apache.flink:flink-clients:jar:1.0-SNAPSHOT in the shaded
jar.
[INFO] Including org.apache.flink:flink-runtime:jar:1.0-SNAPSHOT in the shaded
jar.
[INFO] Excluding io.netty:netty-all:jar:4.0.27.Final from the shaded jar.
[INFO] Including org.javassist:javassist:jar:3.18.2-GA in the shaded jar.
[INFO] Excluding com.typesafe.akka:akka-actor_2.10:jar:2.3.7 from the shaded
jar.
[INFO] Excluding com.typesafe:config:jar:1.2.1 from the shaded jar.
[INFO] Excluding com.typesafe.akka:akka-remote_2.10:jar:2.3.7 from the shaded
jar.
[INFO] Excluding io.netty:netty:jar:3.8.0.Final from the shaded jar.
[INFO] Including com.google.protobuf:protobuf-java:jar:2.5.0 in the shaded jar.
[INFO] Excluding org.uncommons.maths:uncommons-maths:jar:1.2.2a from the shaded
jar.
[INFO] Excluding com.typesafe.akka:akka-slf4j_2.10:jar:2.3.7 from the shaded
jar.
[INFO] Including org.clapper:grizzled-slf4j_2.10:jar:1.0.2 in the shaded jar.
[INFO] Excluding com.github.scopt:scopt_2.10:jar:3.2.0 from the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-core:jar:3.1.0 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-jvm:jar:3.1.0 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-json:jar:3.1.0 in the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-databind:jar:2.4.2 from the
shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-annotations:jar:2.4.0 from
the shaded jar.
[INFO] Excluding com.fasterxml.jackson.core:jackson-core:jar:2.4.2 from the
shaded jar.
[INFO] Including org.apache.zookeeper:zookeeper:jar:3.4.6 in the shaded jar.
[INFO] Including jline:jline:jar:0.9.94 in the shaded jar.
[INFO] Excluding junit:junit:jar:3.8.1 from the shaded jar.
[INFO] Excluding com.twitter:chill_2.10:jar:0.5.2 from the shaded jar.
[INFO] Excluding commons-cli:commons-cli:jar:1.2 from the shaded jar.
[INFO] Including org.apache.flink:flink-connector-kafka-0.8:jar:1.0-SNAPSHOT in
the shaded jar.
[INFO] Including org.apache.flink:flink-connector-kafka-base:jar:1.0-SNAPSHOT
in the shaded jar.
[INFO] Including com.101tec:zkclient:jar:0.7 in the shaded jar.
[INFO] Including org.apache.kafka:kafka_2.10:jar:0.8.2.0 in the shaded jar.
[INFO] Including org.apache.kafka:kafka-clients:jar:0.8.2.0 in the shaded jar.
[INFO] Including net.jpountz.lz4:lz4:jar:1.2.0 in the shaded jar.
[INFO] Including com.yammer.metrics:metrics-core:jar:2.2.0 in the shaded jar.
[INFO] Including org.apache.flink:flink-shaded-curator-recipes:jar:1.0-SNAPSHOT
in the shaded jar.
[WARNING] flink-runtime-1.0-SNAPSHOT.jar, jline-0.9.94.jar,
flink-shaded-curator-recipes-1.0-SNAPSHOT.jar define 35 overlapping classes:
[WARNING] - jline.ANSIBuffer
[WARNING] - jline.ArgumentCompletor$AbstractArgumentDelimiter
[WARNING] - jline.ConsoleReaderInputStream$ConsoleEnumeration
[WARNING] - jline.ConsoleReaderInputStream$ConsoleLineInputStream
[WARNING] - jline.SimpleCompletor$NoOpFilter
[WARNING] - jline.ANSIBuffer$ANSICodes
[WARNING] - jline.ConsoleReader
[WARNING] - jline.CompletionHandler
[WARNING] - jline.CursorBuffer
[WARNING] - jline.ConsoleRunner
[WARNING] - 25 more...
[WARNING] commons-beanutils-1.7.0.jar, commons-beanutils-core-1.8.0.jar define
92 overlapping classes:
[WARNING] - org.apache.commons.beanutils.ConvertUtilsBean
[WARNING] - org.apache.commons.beanutils.converters.SqlTimeConverter
[WARNING] - org.apache.commons.beanutils.Converter
[WARNING] - org.apache.commons.beanutils.converters.FloatArrayConverter
[WARNING] - org.apache.commons.beanutils.NestedNullException
[WARNING] - org.apache.commons.beanutils.ConvertingWrapDynaBean
[WARNING] - org.apache.commons.beanutils.converters.LongArrayConverter
[WARNING] - org.apache.commons.collections.ArrayStack
[WARNING] - org.apache.commons.beanutils.converters.SqlDateConverter
[WARNING] - org.apache.commons.beanutils.converters.BooleanArrayConverter
[WARNING] - 82 more...
[WARNING] flink-runtime-1.0-SNAPSHOT.jar,
flink-shaded-curator-recipes-1.0-SNAPSHOT.jar define 1060 overlapping classes:
[WARNING] -
org.jboss.netty.handler.traffic.TrafficCounter$TrafficMonitoringTask
[WARNING] - org.jboss.netty.handler.codec.spdy.SpdyGoAwayFrame
[WARNING] - org.apache.commons.lang3.SerializationUtils
[WARNING] - org.jboss.netty.util.internal.jzlib.Deflate
[WARNING] - org.jboss.netty.channel.socket.ServerSocketChannelConfig
[WARNING] - org.jboss.netty.util.internal.jzlib.Tree
[WARNING] - org.jboss.netty.handler.codec.serialization.ObjectDecoder
[WARNING] - org.jboss.netty.util.internal.ConcurrentHashMap$HashIterator
[WARNING] -
org.jboss.netty.util.internal.ConcurrentIdentityWeakKeyHashMap$Segment
[WARNING] - org.apache.commons.lang3.LocaleUtils
[WARNING] - 1050 more...
[WARNING] zookeeper-3.4.6.jar, flink-runtime-1.0-SNAPSHOT.jar,
flink-shaded-curator-recipes-1.0-SNAPSHOT.jar define 440 overlapping classes:
[WARNING] - org.apache.zookeeper.server.NettyServerCnxn$DumpCommand
[WARNING] - org.apache.zookeeper.proto.GetMaxChildrenRequest
[WARNING] - org.apache.jute.XmlInputArchive$XMLParser
[WARNING] - org.apache.jute.CsvInputArchive$1
[WARNING] - org.apache.zookeeper.server.NettyServerCnxn$SendBufferWriter
[WARNING] - org.apache.zookeeper.KeeperException$CodeDeprecated
[WARNING] - org.apache.zookeeper.server.ZooKeeperServerMXBean
[WARNING] - org.apache.zookeeper.server.NettyServerCnxn$EnvCommand
[WARNING] - org.apache.zookeeper.server.quorum.Learner$PacketInFlight
[WARNING] - org.apache.zookeeper.server.quorum.LeaderMXBean
[WARNING] - 430 more...
[WARNING] maven-shade-plugin has detected that some class files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the class is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See http://docs.codehaus.org/display/MAVENUSER/Shade+Plugin
[INFO] Replacing original artifact with shaded artifact.
[INFO] Replacing
/home/robert/flink-workdir/debug-1.0-2.11/target/debug-1.0-2.11-1.0-SNAPSHOT.jar
with
/home/robert/flink-workdir/debug-1.0-2.11/target/debug-1.0-2.11-1.0-SNAPSHOT-shaded.jar
[INFO]
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @
debug-1.0-2.11 ---
[INFO] Installing
/home/robert/flink-workdir/debug-1.0-2.11/target/debug-1.0-2.11-1.0-SNAPSHOT.jar
to
/home/robert/.m2/repository/com/dataartisans/debug-1.0-2.11/1.0-SNAPSHOT/debug-1.0-2.11-1.0-SNAPSHOT.jar
[INFO] Installing /home/robert/flink-workdir/debug-1.0-2.11/pom.xml to
/home/robert/.m2/repository/com/dataartisans/debug-1.0-2.11/1.0-SNAPSHOT/debug-1.0-2.11-1.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.248s
[INFO] Finished at: Thu Jan 21 10:08:20 CET 2016
[INFO] Final Memory: 17M/227M
[INFO] ------------------------------------------------------------------------
../apache-maven-3.0.4/bin/mvn clean install 31.31s user 4.26s system 286% cpu
12.423 total
{code}
> Kafka Connector unusable with quickstarts - shading issue
> ---------------------------------------------------------
>
> Key: FLINK-3247
> URL: https://issues.apache.org/jira/browse/FLINK-3247
> Project: Flink
> Issue Type: Bug
> Components: Kafka Connector
> Affects Versions: 1.0.0
> Reporter: Stephan Ewen
> Assignee: Robert Metzger
> Priority: Blocker
> Fix For: 1.0.0
>
>
> The Kafka Connector now requires Curator, which is referenced as
> {{flink-shaded-curator}}. The quickstarts make sure it is not packaged into
> the jar file via exclusions.
> The curator classes are however only in relocated form in the flink-dist.jar
> - relocated manually in the {{flink-runtime}} project. The connector can thus
> not use find the Curator classes and fails.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
