What is the output of the following on the nodes? I have a suspision
that something sneaks in from one of the classpath variables that
Flink picks up:

echo ${HADOOP_CLASSPATH}:${HADOOP_CONF_DIR}:${YARN_CONF_DIR}:${HBASE_CONF_DIR}

On Tue, Nov 29, 2016 at 9:17 PM, Wayne Collins <wayn...@dades.ca> wrote:
> Hi Max,
>
> I rebuilt my sandbox with Beam 0.3.0-incubating and Flink 1.1.2 and I'm
> still seeing the following error message with the StreamWordCount demo code:
>
> Caused by: java.lang.IllegalAccessError: tried to access method
> com.google.common.base.Optional.<init>()V from class
> com.google.common.base.Absent
>         at com.google.common.base.Absent.<init>(Absent.java:35)
>         at com.google.common.base.Absent.<clinit>(Absent.java:33)
>         at sun.misc.Unsafe.ensureClassInitialized(Native Method)
> ...
>
> The full mvn dependency tree is below this msg but here is the grep of guava
> from that.
> It looks to me like maven is properly identifying/handling the duplicates
> and conflicts:
>
> beam-starter$ mvn dependency:tree -Dverbose | grep guava
> [INFO] |  |  |  +- (com.google.guava:guava:jar:19.0:compile - omitted for
> duplicate)
> [INFO] |  |  |  |  +- (com.google.guava:guava:jar:19.0:runtime - omitted for
> duplicate)
> [INFO] |  |  |  |  |  +- (com.google.guava:guava:jar:18.0:runtime - omitted
> for conflict with 19.0)
> [INFO] |  |  |  |  +- (com.google.guava:guava:jar:19.0:runtime - omitted for
> duplicate)
> [INFO] |  |  |  +- (com.google.guava:guava:jar:19.0:runtime - omitted for
> duplicate)
> [INFO] |  |  |  +- (com.google.guava:guava:jar:18.0:compile - omitted for
> conflict with 19.0)
> [INFO] |  |  |  +- (com.google.guava:guava:jar:18.0:compile - omitted for
> conflict with 19.0)
> [INFO]    +- com.google.guava:guava:jar:19.0:compile
>
> Is there anything else I can check?
>
> Thanks!
> Wayne
>
> ---------------------------------------------------------------
> Full mvn dependency tree:
> beam-starter$ mvn dependency:tree -Dverbose
> [INFO] Scanning for projects...
> [INFO]
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Building beam-starter 0.2
> [INFO]
> ------------------------------------------------------------------------
> [INFO]
> [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ beam-starter ---
> [INFO] com.dataradiant.beam:beam-starter:jar:0.2
> [INFO] +-
> org.apache.beam:beam-runners-flink_2.10:jar:0.3.0-incubating:compile
> [INFO] |  +- org.apache.flink:flink-streaming-java_2.10:jar:1.1.2:compile
> [INFO] |  |  +- org.apache.flink:flink-core:jar:1.1.2:compile
> [INFO] |  |  |  +- org.apache.flink:flink-annotations:jar:1.1.2:compile
> [INFO] |  |  |  |  +- (com.google.code.findbugs:jsr305:jar:1.3.9:compile -
> omitted for duplicate)
> [INFO] |  |  |  |  +- (org.apache.commons:commons-lang3:jar:3.3.2:compile -
> omitted for duplicate)
> [INFO] |  |  |  |  +- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  |  +- (org.slf4j:slf4j-log4j12:jar:1.7.7:compile - omitted
> for duplicate)
> [INFO] |  |  |  |  +- (log4j:log4j:jar:1.2.17:compile - omitted for
> duplicate)
> [INFO] |  |  |  |  \- (org.apache.flink:force-shading:jar:1.1.2:compile -
> omitted for duplicate)
> [INFO] |  |  |  +- org.apache.flink:flink-metrics-core:jar:1.1.2:compile
> [INFO] |  |  |  |  +- (com.google.code.findbugs:jsr305:jar:1.3.9:compile -
> omitted for duplicate)
> [INFO] |  |  |  |  +- (org.apache.commons:commons-lang3:jar:3.3.2:compile -
> omitted for duplicate)
> [INFO] |  |  |  |  +- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  |  +- (org.slf4j:slf4j-log4j12:jar:1.7.7:compile - omitted
> for duplicate)
> [INFO] |  |  |  |  +- (log4j:log4j:jar:1.2.17:compile - omitted for
> duplicate)
> [INFO] |  |  |  |  \- (org.apache.flink:force-shading:jar:1.1.2:compile -
> omitted for duplicate)
> [INFO] |  |  |  +- com.esotericsoftware.kryo:kryo:jar:2.24.0:compile
> [INFO] |  |  |  |  +- com.esotericsoftware.minlog:minlog:jar:1.2:compile
> [INFO] |  |  |  |  \- org.objenesis:objenesis:jar:2.1:compile
> [INFO] |  |  |  +- (org.apache.avro:avro:jar:1.7.6:compile - omitted for
> conflict with 1.8.1)
> [INFO] |  |  |  +- (org.apache.flink:flink-shaded-hadoop2:jar:1.1.2:compile
> - omitted for duplicate)
> [INFO] |  |  |  +- (com.google.code.findbugs:jsr305:jar:1.3.9:compile -
> omitted for duplicate)
> [INFO] |  |  |  +- (org.apache.commons:commons-lang3:jar:3.3.2:compile -
> omitted for duplicate)
> [INFO] |  |  |  +- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- (org.slf4j:slf4j-log4j12:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- (log4j:log4j:jar:1.2.17:compile - omitted for duplicate)
> [INFO] |  |  |  \- (org.apache.flink:force-shading:jar:1.1.2:compile -
> omitted for duplicate)
> [INFO] |  |  +- org.apache.flink:flink-runtime_2.10:jar:1.1.2:compile
> [INFO] |  |  |  +- (org.apache.flink:flink-core:jar:1.1.2:compile - omitted
> for duplicate)
> [INFO] |  |  |  +- (org.apache.flink:flink-java:jar:1.1.2:compile - omitted
> for duplicate)
> [INFO] |  |  |  +- (org.apache.flink:flink-shaded-hadoop2:jar:1.1.2:compile
> - omitted for duplicate)
> [INFO] |  |  |  +- (commons-cli:commons-cli:jar:1.3.1:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- io.netty:netty-all:jar:4.0.27.Final:compile
> [INFO] |  |  |  +- org.javassist:javassist:jar:3.18.2-GA:compile
> [INFO] |  |  |  +- org.scala-lang:scala-library:jar:2.10.4:compile
> [INFO] |  |  |  +- com.typesafe.akka:akka-actor_2.10:jar:2.3.7:compile
> [INFO] |  |  |  |  +- (org.scala-lang:scala-library:jar:2.10.4:compile -
> omitted for duplicate)
> [INFO] |  |  |  |  \- com.typesafe:config:jar:1.2.1:compile
> [INFO] |  |  |  +- com.typesafe.akka:akka-remote_2.10:jar:2.3.7:compile
> [INFO] |  |  |  |  +- (org.scala-lang:scala-library:jar:2.10.4:compile -
> omitted for duplicate)
> [INFO] |  |  |  |  +- (com.typesafe.akka:akka-actor_2.10:jar:2.3.7:compile -
> omitted for duplicate)
> [INFO] |  |  |  |  +- io.netty:netty:jar:3.8.0.Final:compile
> [INFO] |  |  |  |  +- (com.google.protobuf:protobuf-java:jar:2.5.0:compile -
> omitted for conflict with 3.0.0)
> [INFO] |  |  |  |  \- org.uncommons.maths:uncommons-maths:jar:1.2.2a:compile
> [INFO] |  |  |  +- com.typesafe.akka:akka-slf4j_2.10:jar:2.3.7:compile
> [INFO] |  |  |  |  +- (org.scala-lang:scala-library:jar:2.10.4:compile -
> omitted for duplicate)
> [INFO] |  |  |  |  +- (com.typesafe.akka:akka-actor_2.10:jar:2.3.7:compile -
> omitted for duplicate)
> [INFO] |  |  |  |  \- (org.slf4j:slf4j-api:jar:1.7.5:compile - omitted for
> conflict with 1.7.7)
> [INFO] |  |  |  +- org.clapper:grizzled-slf4j_2.10:jar:1.0.2:compile
> [INFO] |  |  |  |  +- (org.scala-lang:scala-library:jar:2.10.3:compile -
> omitted for conflict with 2.10.4)
> [INFO] |  |  |  |  \- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- com.github.scopt:scopt_2.10:jar:3.2.0:compile
> [INFO] |  |  |  +- io.dropwizard.metrics:metrics-core:jar:3.1.0:compile
> [INFO] |  |  |  |  \- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- io.dropwizard.metrics:metrics-jvm:jar:3.1.0:compile
> [INFO] |  |  |  |  +- (io.dropwizard.metrics:metrics-core:jar:3.1.0:compile
> - omitted for duplicate)
> [INFO] |  |  |  |  \- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- io.dropwizard.metrics:metrics-json:jar:3.1.0:compile
> [INFO] |  |  |  |  +- (io.dropwizard.metrics:metrics-core:jar:3.1.0:compile
> - omitted for duplicate)
> [INFO] |  |  |  |  +-
> (com.fasterxml.jackson.core:jackson-databind:jar:2.4.2:compile - omitted for
> conflict with 2.7.2)
> [INFO] |  |  |  |  \- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- org.apache.zookeeper:zookeeper:jar:3.4.6:compile
> [INFO] |  |  |  |  +- (org.slf4j:slf4j-api:jar:1.6.1:compile - omitted for
> conflict with 1.7.7)
> [INFO] |  |  |  |  +- (org.slf4j:slf4j-log4j12:jar:1.6.1:compile - omitted
> for conflict with 1.7.7)
> [INFO] |  |  |  |  +- (log4j:log4j:jar:1.2.16:compile - omitted for conflict
> with 1.2.17)
> [INFO] |  |  |  |  +- jline:jline:jar:0.9.94:compile
> [INFO] |  |  |  |  |  \- junit:junit:jar:3.8.1:compile
> [INFO] |  |  |  |  \- (io.netty:netty:jar:3.7.0.Final:compile - omitted for
> conflict with 3.8.0.Final)
> [INFO] |  |  |  +- com.twitter:chill_2.10:jar:0.7.4:compile
> [INFO] |  |  |  |  +- (org.scala-lang:scala-library:jar:2.10.5:compile -
> omitted for conflict with 2.10.4)
> [INFO] |  |  |  |  +- com.twitter:chill-java:jar:0.7.4:compile
> [INFO] |  |  |  |  |  \- (com.esotericsoftware.kryo:kryo:jar:2.21:compile -
> omitted for conflict with 2.24.0)
> [INFO] |  |  |  |  \- (com.esotericsoftware.kryo:kryo:jar:2.21:compile -
> omitted for conflict with 2.24.0)
> [INFO] |  |  |  +- (com.google.code.findbugs:jsr305:jar:1.3.9:compile -
> omitted for duplicate)
> [INFO] |  |  |  +- (org.apache.commons:commons-lang3:jar:3.3.2:compile -
> omitted for duplicate)
> [INFO] |  |  |  +- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- (org.slf4j:slf4j-log4j12:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- (log4j:log4j:jar:1.2.17:compile - omitted for duplicate)
> [INFO] |  |  |  \- (org.apache.flink:force-shading:jar:1.1.2:compile -
> omitted for duplicate)
> [INFO] |  |  +- (org.apache.flink:flink-clients_2.10:jar:1.1.2:compile -
> omitted for duplicate)
> [INFO] |  |  +- org.apache.commons:commons-math3:jar:3.5:compile
> [INFO] |  |  +-
> org.apache.sling:org.apache.sling.commons.json:jar:2.0.6:compile
> [INFO] |  |  +- com.google.code.findbugs:jsr305:jar:1.3.9:compile
> [INFO] |  |  +- org.apache.commons:commons-lang3:jar:3.3.2:compile
> [INFO] |  |  +- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> conflict with 1.7.14)
> [INFO] |  |  +- org.slf4j:slf4j-log4j12:jar:1.7.7:compile
> [INFO] |  |  |  +- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  \- (log4j:log4j:jar:1.2.17:compile - omitted for duplicate)
> [INFO] |  |  +- log4j:log4j:jar:1.2.17:compile
> [INFO] |  |  \- org.apache.flink:force-shading:jar:1.1.2:compile
> [INFO] |  +- org.apache.flink:flink-java:jar:1.1.2:compile
> [INFO] |  |  +- (org.apache.flink:flink-core:jar:1.1.2:compile - omitted for
> duplicate)
> [INFO] |  |  +- org.apache.flink:flink-shaded-hadoop2:jar:1.1.2:compile
> [INFO] |  |  |  +- (commons-cli:commons-cli:jar:1.3.1:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- (org.apache.commons:commons-math3:jar:3.5:compile -
> omitted for duplicate)
> [INFO] |  |  |  +- xmlenc:xmlenc:jar:0.52:compile
> [INFO] |  |  |  +- commons-codec:commons-codec:jar:1.4:compile
> [INFO] |  |  |  +- commons-io:commons-io:jar:2.4:compile
> [INFO] |  |  |  +- commons-net:commons-net:jar:3.1:compile
> [INFO] |  |  |  +- commons-collections:commons-collections:jar:3.2.1:compile
> [INFO] |  |  |  +- javax.servlet:servlet-api:jar:2.5:compile
> [INFO] |  |  |  +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
> [INFO] |  |  |  +- com.sun.jersey:jersey-core:jar:1.9:compile
> [INFO] |  |  |  +- commons-el:commons-el:jar:1.0:runtime
> [INFO] |  |  |  |  \- (commons-logging:commons-logging:jar:1.0.3:runtime -
> omitted for conflict with 1.1.3)
> [INFO] |  |  |  +- commons-logging:commons-logging:jar:1.1.3:compile
> [INFO] |  |  |  +- com.jamesmurty.utils:java-xmlbuilder:jar:0.4:compile
> [INFO] |  |  |  +- commons-lang:commons-lang:jar:2.6:compile
> [INFO] |  |  |  +-
> commons-configuration:commons-configuration:jar:1.7:compile
> [INFO] |  |  |  |  +-
> (commons-collections:commons-collections:jar:3.2.1:compile - omitted for
> duplicate)
> [INFO] |  |  |  |  +- (commons-lang:commons-lang:jar:2.6:compile - omitted
> for duplicate)
> [INFO] |  |  |  |  +- (commons-logging:commons-logging:jar:1.1.1:compile -
> omitted for conflict with 1.1.3)
> [INFO] |  |  |  |  \- (commons-digester:commons-digester:jar:1.8.1:compile -
> omitted for duplicate)
> [INFO] |  |  |  +- commons-digester:commons-digester:jar:1.8.1:compile
> [INFO] |  |  |  |  \- (commons-logging:commons-logging:jar:1.1.1:compile -
> omitted for conflict with 1.1.3)
> [INFO] |  |  |  +- org.codehaus.jackson:jackson-core-asl:jar:1.8.8:compile
> [INFO] |  |  |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile
> [INFO] |  |  |  |  \-
> (org.codehaus.jackson:jackson-core-asl:jar:1.8.8:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- (org.apache.avro:avro:jar:1.7.6:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
> [INFO] |  |  |  +- (org.xerial.snappy:snappy-java:jar:1.0.5:compile -
> omitted for conflict with 1.1.2.1)
> [INFO] |  |  |  +- com.jcraft:jsch:jar:0.1.42:compile
> [INFO] |  |  |  +- (org.apache.zookeeper:zookeeper:jar:3.4.6:compile -
> omitted for duplicate)
> [INFO] |  |  |  +- (org.apache.commons:commons-compress:jar:1.4.1:compile -
> omitted for conflict with 1.9)
> [INFO] |  |  |  +- org.tukaani:xz:jar:1.0:compile
> [INFO] |  |  |  +-
> commons-beanutils:commons-beanutils-bean-collections:jar:1.8.3:compile
> [INFO] |  |  |  |  \- (commons-logging:commons-logging:jar:1.1.1:compile -
> omitted for conflict with 1.1.3)
> [INFO] |  |  |  +- commons-daemon:commons-daemon:jar:1.0.13:compile
> [INFO] |  |  |  +- javax.xml.bind:jaxb-api:jar:2.2.2:compile
> [INFO] |  |  |  |  +- (javax.xml.stream:stax-api:jar:1.0-2:compile - omitted
> for duplicate)
> [INFO] |  |  |  |  \- (javax.activation:activation:jar:1.1:compile - omitted
> for duplicate)
> [INFO] |  |  |  +- javax.xml.stream:stax-api:jar:1.0-2:compile
> [INFO] |  |  |  +- javax.activation:activation:jar:1.1:compile
> [INFO] |  |  |  +- com.google.inject:guice:jar:3.0:compile
> [INFO] |  |  |  |  +- (javax.inject:javax.inject:jar:1:compile - omitted for
> duplicate)
> [INFO] |  |  |  |  \- (aopalliance:aopalliance:jar:1.0:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- javax.inject:javax.inject:jar:1:compile
> [INFO] |  |  |  +- aopalliance:aopalliance:jar:1.0:compile
> [INFO] |  |  |  +- (org.apache.flink:force-shading:jar:1.1.2:compile -
> omitted for duplicate)
> [INFO] |  |  |  +- (org.apache.commons:commons-lang3:jar:3.3.2:compile -
> omitted for duplicate)
> [INFO] |  |  |  +- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- (org.slf4j:slf4j-log4j12:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  \- (log4j:log4j:jar:1.2.17:compile - omitted for duplicate)
> [INFO] |  |  +- (org.apache.commons:commons-math3:jar:3.5:compile - omitted
> for duplicate)
> [INFO] |  |  +- (com.google.code.findbugs:jsr305:jar:1.3.9:compile - omitted
> for duplicate)
> [INFO] |  |  +- (org.apache.commons:commons-lang3:jar:3.3.2:compile -
> omitted for duplicate)
> [INFO] |  |  +- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  +- (org.slf4j:slf4j-log4j12:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  +- (log4j:log4j:jar:1.2.17:compile - omitted for duplicate)
> [INFO] |  |  \- (org.apache.flink:force-shading:jar:1.1.2:compile - omitted
> for duplicate)
> [INFO] |  +- org.apache.flink:flink-clients_2.10:jar:1.1.2:compile
> [INFO] |  |  +- (org.apache.flink:flink-core:jar:1.1.2:compile - omitted for
> duplicate)
> [INFO] |  |  +- (org.apache.flink:flink-runtime_2.10:jar:1.1.2:compile -
> omitted for duplicate)
> [INFO] |  |  +- org.apache.flink:flink-optimizer_2.10:jar:1.1.2:compile
> [INFO] |  |  |  +- (org.apache.flink:flink-core:jar:1.1.2:compile - omitted
> for duplicate)
> [INFO] |  |  |  +- (org.apache.flink:flink-runtime_2.10:jar:1.1.2:compile -
> omitted for duplicate)
> [INFO] |  |  |  +- (org.apache.flink:flink-java:jar:1.1.2:compile - omitted
> for duplicate)
> [INFO] |  |  |  +- (com.google.code.findbugs:jsr305:jar:1.3.9:compile -
> omitted for duplicate)
> [INFO] |  |  |  +- (org.apache.commons:commons-lang3:jar:3.3.2:compile -
> omitted for duplicate)
> [INFO] |  |  |  +- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- (org.slf4j:slf4j-log4j12:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  |  +- (log4j:log4j:jar:1.2.17:compile - omitted for duplicate)
> [INFO] |  |  |  \- (org.apache.flink:force-shading:jar:1.1.2:compile -
> omitted for duplicate)
> [INFO] |  |  +- (org.apache.flink:flink-java:jar:1.1.2:compile - omitted for
> duplicate)
> [INFO] |  |  +- commons-cli:commons-cli:jar:1.3.1:compile
> [INFO] |  |  +- (com.google.code.findbugs:jsr305:jar:1.3.9:compile - omitted
> for duplicate)
> [INFO] |  |  +- (org.apache.commons:commons-lang3:jar:3.3.2:compile -
> omitted for duplicate)
> [INFO] |  |  +- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  +- (org.slf4j:slf4j-log4j12:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  +- (log4j:log4j:jar:1.2.17:compile - omitted for duplicate)
> [INFO] |  |  \- (org.apache.flink:force-shading:jar:1.1.2:compile - omitted
> for duplicate)
> [INFO] |  +-
> org.apache.beam:beam-sdks-java-core:jar:0.3.0-incubating:compile
> [INFO] |  |  +- io.grpc:grpc-auth:jar:1.0.1:compile
> [INFO] |  |  |  +-
> com.google.auth:google-auth-library-credentials:jar:0.4.0:compile
> [INFO] |  |  |  \- (io.grpc:grpc-core:jar:1.0.1:compile - omitted for
> duplicate)
> [INFO] |  |  +- io.grpc:grpc-core:jar:1.0.1:compile
> [INFO] |  |  |  +- (com.google.guava:guava:jar:19.0:compile - omitted for
> duplicate)
> [INFO] |  |  |  \- io.grpc:grpc-context:jar:1.0.1:compile
> [INFO] |  |  +- io.grpc:grpc-netty:jar:1.0.1:compile
> [INFO] |  |  |  +- io.netty:netty-codec-http2:jar:4.1.3.Final:compile
> [INFO] |  |  |  |  +- io.netty:netty-codec-http:jar:4.1.3.Final:compile
> [INFO] |  |  |  |  |  \- (io.netty:netty-codec:jar:4.1.3.Final:compile -
> omitted for duplicate)
> [INFO] |  |  |  |  \- (io.netty:netty-handler:jar:4.1.3.Final:compile -
> omitted for duplicate)
> [INFO] |  |  |  \- (io.grpc:grpc-core:jar:1.0.1:compile - omitted for
> duplicate)
> [INFO] |  |  +- io.grpc:grpc-stub:jar:1.0.1:compile
> [INFO] |  |  |  \- (io.grpc:grpc-core:jar:1.0.1:compile - omitted for
> duplicate)
> [INFO] |  |  +- io.grpc:grpc-all:jar:1.0.1:runtime
> [INFO] |  |  |  +- (io.grpc:grpc-auth:jar:1.0.1:runtime - omitted for
> duplicate)
> [INFO] |  |  |  +- (io.grpc:grpc-context:jar:1.0.1:runtime - omitted for
> duplicate)
> [INFO] |  |  |  +- io.grpc:grpc-protobuf:jar:1.0.1:runtime
> [INFO] |  |  |  |  +- (com.google.guava:guava:jar:19.0:runtime - omitted for
> duplicate)
> [INFO] |  |  |  |  +- (com.google.protobuf:protobuf-java:jar:3.0.0:runtime -
> omitted for conflict with 2.5.0)
> [INFO] |  |  |  |  +-
> com.google.protobuf:protobuf-java-util:jar:3.0.0:runtime
> [INFO] |  |  |  |  |  +-
> (com.google.protobuf:protobuf-java:jar:3.0.0:runtime - omitted for conflict
> with 2.5.0)
> [INFO] |  |  |  |  |  +- (com.google.guava:guava:jar:18.0:runtime - omitted
> for conflict with 19.0)
> [INFO] |  |  |  |  |  \- com.google.code.gson:gson:jar:2.3:runtime
> [INFO] |  |  |  |  +- (io.grpc:grpc-core:jar:1.0.1:runtime - omitted for
> duplicate)
> [INFO] |  |  |  |  \- (io.grpc:grpc-protobuf-lite:jar:1.0.1:runtime -
> omitted for duplicate)
> [INFO] |  |  |  +- (io.grpc:grpc-netty:jar:1.0.1:runtime - omitted for
> duplicate)
> [INFO] |  |  |  +- (io.grpc:grpc-stub:jar:1.0.1:runtime - omitted for
> duplicate)
> [INFO] |  |  |  +- io.grpc:grpc-protobuf-nano:jar:1.0.1:runtime
> [INFO] |  |  |  |  +- (com.google.guava:guava:jar:19.0:runtime - omitted for
> duplicate)
> [INFO] |  |  |  |  +-
> com.google.protobuf.nano:protobuf-javanano:jar:3.0.0-alpha-5:runtime
> [INFO] |  |  |  |  \- (io.grpc:grpc-core:jar:1.0.1:runtime - omitted for
> duplicate)
> [INFO] |  |  |  +- (io.grpc:grpc-core:jar:1.0.1:runtime - omitted for
> duplicate)
> [INFO] |  |  |  +- io.grpc:grpc-okhttp:jar:1.0.1:runtime
> [INFO] |  |  |  |  +- com.squareup.okio:okio:jar:1.6.0:runtime
> [INFO] |  |  |  |  +- com.squareup.okhttp:okhttp:jar:2.5.0:runtime
> [INFO] |  |  |  |  |  \- (com.squareup.okio:okio:jar:1.6.0:runtime - omitted
> for duplicate)
> [INFO] |  |  |  |  \- (io.grpc:grpc-core:jar:1.0.1:runtime - omitted for
> duplicate)
> [INFO] |  |  |  \- (io.grpc:grpc-protobuf-lite:jar:1.0.1:runtime - omitted
> for duplicate)
> [INFO] |  |  +- io.grpc:grpc-protobuf-lite:jar:1.0.1:runtime
> [INFO] |  |  |  +- (com.google.guava:guava:jar:19.0:runtime - omitted for
> duplicate)
> [INFO] |  |  |  +- (com.google.protobuf:protobuf-lite:jar:3.0.1:runtime -
> omitted for duplicate)
> [INFO] |  |  |  \- (io.grpc:grpc-core:jar:1.0.1:runtime - omitted for
> duplicate)
> [INFO] |  |  +- com.google.protobuf:protobuf-lite:jar:3.0.1:runtime
> [INFO] |  |  +-
> com.google.auth:google-auth-library-oauth2-http:jar:0.4.0:compile
> [INFO] |  |  |  +-
> (com.google.auth:google-auth-library-credentials:jar:0.4.0:compile - omitted
> for duplicate)
> [INFO] |  |  |  +-
> (com.google.http-client:google-http-client:jar:1.19.0:compile - omitted for
> conflict with 1.22.0)
> [INFO] |  |  |  \-
> (com.google.http-client:google-http-client-jackson2:jar:1.19.0:compile -
> omitted for conflict with 1.22.0)
> [INFO] |  |  +- io.netty:netty-handler:jar:4.1.3.Final:compile
> [INFO] |  |  |  +- io.netty:netty-buffer:jar:4.1.3.Final:compile
> [INFO] |  |  |  |  \- io.netty:netty-common:jar:4.1.3.Final:compile
> [INFO] |  |  |  +- io.netty:netty-transport:jar:4.1.3.Final:compile
> [INFO] |  |  |  |  +- (io.netty:netty-buffer:jar:4.1.3.Final:compile -
> omitted for duplicate)
> [INFO] |  |  |  |  \- io.netty:netty-resolver:jar:4.1.3.Final:compile
> [INFO] |  |  |  |     \- (io.netty:netty-common:jar:4.1.3.Final:compile -
> omitted for duplicate)
> [INFO] |  |  |  \- io.netty:netty-codec:jar:4.1.3.Final:compile
> [INFO] |  |  |     \- (io.netty:netty-transport:jar:4.1.3.Final:compile -
> omitted for duplicate)
> [INFO] |  |  +- com.google.api.grpc:grpc-google-pubsub-v1:jar:0.1.0:compile
> [INFO] |  |  |  +- (com.google.protobuf:protobuf-java:jar:3.0.0:compile -
> omitted for duplicate)
> [INFO] |  |  |  +-
> com.google.api.grpc:grpc-google-common-protos:jar:0.1.0:compile
> [INFO] |  |  |  |  \- (com.google.protobuf:protobuf-java:jar:3.0.0:compile -
> omitted for duplicate)
> [INFO] |  |  |  \- com.google.api.grpc:grpc-google-iam-v1:jar:0.1.0:compile
> [INFO] |  |  |     \- (com.google.protobuf:protobuf-java:jar:3.0.0:compile -
> omitted for duplicate)
> [INFO] |  |  +- com.google.api-client:google-api-client:jar:1.22.0:compile
> [INFO] |  |  |  +-
> (com.google.oauth-client:google-oauth-client:jar:1.22.0:compile - omitted
> for duplicate)
> [INFO] |  |  |  \-
> (com.google.http-client:google-http-client-jackson2:jar:1.22.0:compile -
> omitted for conflict with 1.19.0)
> [INFO] |  |  +-
> com.google.apis:google-api-services-bigquery:jar:v2-rev295-1.22.0:compile
> [INFO] |  |  |  \-
> (com.google.api-client:google-api-client:jar:1.22.0:compile - omitted for
> duplicate)
> [INFO] |  |  +-
> com.google.apis:google-api-services-cloudresourcemanager:jar:v1-rev6-1.22.0:compile
> [INFO] |  |  |  \-
> (com.google.api-client:google-api-client:jar:1.22.0:compile - omitted for
> duplicate)
> [INFO] |  |  +-
> com.google.apis:google-api-services-pubsub:jar:v1-rev10-1.22.0:compile
> [INFO] |  |  |  \-
> (com.google.api-client:google-api-client:jar:1.22.0:compile - omitted for
> duplicate)
> [INFO] |  |  +-
> com.google.apis:google-api-services-storage:jar:v1-rev71-1.22.0:compile
> [INFO] |  |  |  \-
> (com.google.api-client:google-api-client:jar:1.22.0:compile - omitted for
> duplicate)
> [INFO] |  |  +- com.google.http-client:google-http-client:jar:1.22.0:compile
> [INFO] |  |  |  \- org.apache.httpcomponents:httpclient:jar:4.0.1:compile
> [INFO] |  |  |     +- org.apache.httpcomponents:httpcore:jar:4.0.1:compile
> [INFO] |  |  |     +- (commons-logging:commons-logging:jar:1.1.1:compile -
> omitted for conflict with 1.1.3)
> [INFO] |  |  |     \- (commons-codec:commons-codec:jar:1.3:compile - omitted
> for conflict with 1.4)
> [INFO] |  |  +-
> com.google.http-client:google-http-client-jackson:jar:1.22.0:runtime
> [INFO] |  |  |  \-
> (com.google.http-client:google-http-client:jar:1.22.0:runtime - omitted for
> duplicate)
> [INFO] |  |  +-
> com.google.http-client:google-http-client-jackson2:jar:1.22.0:compile
> [INFO] |  |  |  +-
> (com.google.http-client:google-http-client:jar:1.22.0:compile - omitted for
> duplicate)
> [INFO] |  |  |  \-
> (com.fasterxml.jackson.core:jackson-core:jar:2.1.3:compile - omitted for
> conflict with 2.7.2)
> [INFO] |  |  +-
> com.google.http-client:google-http-client-protobuf:jar:1.22.0:runtime
> [INFO] |  |  |  +-
> (com.google.http-client:google-http-client:jar:1.22.0:runtime - omitted for
> duplicate)
> [INFO] |  |  |  \- (com.google.protobuf:protobuf-java:jar:2.6.1:runtime -
> omitted for conflict with 3.0.0)
> [INFO] |  |  +-
> com.google.oauth-client:google-oauth-client-java6:jar:1.22.0:compile
> [INFO] |  |  |  \-
> (com.google.oauth-client:google-oauth-client:jar:1.22.0:compile - omitted
> for duplicate)
> [INFO] |  |  +-
> com.google.oauth-client:google-oauth-client:jar:1.22.0:compile
> [INFO] |  |  |  \-
> (com.google.http-client:google-http-client:jar:1.22.0:compile - omitted for
> duplicate)
> [INFO] |  |  +- com.google.cloud.bigdataoss:gcsio:jar:1.4.5:compile
> [INFO] |  |  |  +-
> com.google.api-client:google-api-client-java6:jar:1.20.0:compile
> [INFO] |  |  |  |  +-
> (com.google.api-client:google-api-client:jar:1.20.0:compile - omitted for
> conflict with 1.22.0)
> [INFO] |  |  |  |  \-
> (com.google.oauth-client:google-oauth-client-java6:jar:1.20.0:compile -
> omitted for conflict with 1.22.0)
> [INFO] |  |  |  +-
> com.google.api-client:google-api-client-jackson2:jar:1.20.0:compile
> [INFO] |  |  |  |  +-
> (com.google.api-client:google-api-client:jar:1.20.0:compile - omitted for
> conflict with 1.22.0)
> [INFO] |  |  |  |  \-
> (com.google.http-client:google-http-client-jackson2:jar:1.20.0:compile -
> omitted for conflict with 1.22.0)
> [INFO] |  |  |  +-
> (com.google.apis:google-api-services-storage:jar:v1-rev35-1.20.0:compile -
> omitted for conflict with v1-rev71-1.22.0)
> [INFO] |  |  |  +- (com.google.guava:guava:jar:18.0:compile - omitted for
> conflict with 19.0)
> [INFO] |  |  |  +-
> (com.google.oauth-client:google-oauth-client:jar:1.20.0:compile - omitted
> for conflict with 1.22.0)
> [INFO] |  |  |  +-
> (com.google.oauth-client:google-oauth-client-java6:jar:1.20.0:compile -
> omitted for conflict with 1.22.0)
> [INFO] |  |  |  \- (com.google.cloud.bigdataoss:util:jar:1.4.5:compile -
> omitted for duplicate)
> [INFO] |  |  +- com.google.cloud.bigdataoss:util:jar:1.4.5:compile
> [INFO] |  |  |  +-
> (com.google.api-client:google-api-client-java6:jar:1.20.0:compile - omitted
> for duplicate)
> [INFO] |  |  |  +-
> (com.google.api-client:google-api-client-jackson2:jar:1.20.0:compile -
> omitted for duplicate)
> [INFO] |  |  |  +-
> (com.google.apis:google-api-services-storage:jar:v1-rev35-1.20.0:compile -
> omitted for conflict with v1-rev71-1.22.0)
> [INFO] |  |  |  +- (com.google.guava:guava:jar:18.0:compile - omitted for
> conflict with 19.0)
> [INFO] |  |  |  +-
> (com.google.oauth-client:google-oauth-client:jar:1.20.0:compile - omitted
> for conflict with 1.22.0)
> [INFO] |  |  |  \-
> (com.google.oauth-client:google-oauth-client-java6:jar:1.20.0:compile -
> omitted for conflict with 1.22.0)
> [INFO] |  |  +- com.google.protobuf:protobuf-java:jar:3.0.0:compile
> [INFO] |  |  +- (com.google.code.findbugs:annotations:jar:3.0.1:compile -
> omitted for duplicate)
> [INFO] |  |  +- com.fasterxml.jackson.core:jackson-core:jar:2.7.2:compile
> [INFO] |  |  +-
> (com.fasterxml.jackson.core:jackson-annotations:jar:2.7.2:compile - omitted
> for duplicate)
> [INFO] |  |  +-
> com.fasterxml.jackson.core:jackson-databind:jar:2.7.2:compile
> [INFO] |  |  |  +-
> (com.fasterxml.jackson.core:jackson-annotations:jar:2.7.0:compile - omitted
> for conflict with 2.7.2)
> [INFO] |  |  |  \-
> (com.fasterxml.jackson.core:jackson-core:jar:2.7.2:compile - omitted for
> duplicate)
> [INFO] |  |  +- (org.slf4j:slf4j-api:jar:1.7.14:compile - omitted for
> conflict with 1.7.7)
> [INFO] |  |  +- net.bytebuddy:byte-buddy:jar:1.4.3:compile
> [INFO] |  |  +- org.apache.avro:avro:jar:1.8.1:compile
> [INFO] |  |  |  +- (org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
> - omitted for conflict with 1.8.8)
> [INFO] |  |  |  +-
> (org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile - omitted for
> conflict with 1.8.8)
> [INFO] |  |  |  +- (com.thoughtworks.paranamer:paranamer:jar:2.7:compile -
> omitted for conflict with 2.3)
> [INFO] |  |  |  +- (org.xerial.snappy:snappy-java:jar:1.1.1.3:compile -
> omitted for conflict with 1.0.5)
> [INFO] |  |  |  +- (org.apache.commons:commons-compress:jar:1.8.1:compile -
> omitted for conflict with 1.4.1)
> [INFO] |  |  |  +- (org.tukaani:xz:jar:1.5:compile - omitted for conflict
> with 1.0)
> [INFO] |  |  |  \- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |  |  +- org.xerial.snappy:snappy-java:jar:1.1.2.1:compile
> [INFO] |  |  +- org.apache.commons:commons-compress:jar:1.9:compile
> [INFO] |  |  \- (joda-time:joda-time:jar:2.4:compile - omitted for
> duplicate)
> [INFO] |  \-
> org.apache.beam:beam-runners-core-java:jar:0.3.0-incubating:compile
> [INFO] |     +-
> (org.apache.beam:beam-sdks-java-core:jar:0.3.0-incubating:compile - omitted
> for duplicate)
> [INFO] |     +-
> (com.fasterxml.jackson.core:jackson-annotations:jar:2.7.2:compile - omitted
> for duplicate)
> [INFO] |     +- (com.google.code.findbugs:annotations:jar:3.0.1:compile -
> omitted for duplicate)
> [INFO] |     +- (joda-time:joda-time:jar:2.4:compile - omitted for
> duplicate)
> [INFO] |     \- (org.slf4j:slf4j-api:jar:1.7.14:compile - omitted for
> conflict with 1.7.7)
> [INFO] +-
> org.apache.beam:beam-runners-flink_2.10-examples:jar:0.3.0-incubating:compile
> [INFO] |  +-
> (org.apache.beam:beam-runners-flink_2.10:jar:0.3.0-incubating:compile -
> omitted for duplicate)
> [INFO] |  \-
> org.apache.flink:flink-connector-kafka-0.8_2.10:jar:1.1.2:compile
> [INFO] |     +-
> org.apache.flink:flink-connector-kafka-base_2.10:jar:1.1.2:compile
> [INFO] |     |  +- (org.apache.kafka:kafka_2.10:jar:0.8.2.2:compile -
> omitted for duplicate)
> [INFO] |     |  +- (com.google.code.findbugs:jsr305:jar:1.3.9:compile -
> omitted for duplicate)
> [INFO] |     |  +- (org.apache.commons:commons-lang3:jar:3.3.2:compile -
> omitted for duplicate)
> [INFO] |     |  +- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |     |  +- (org.slf4j:slf4j-log4j12:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |     |  +- (log4j:log4j:jar:1.2.17:compile - omitted for duplicate)
> [INFO] |     |  \- (org.apache.flink:force-shading:jar:1.1.2:compile -
> omitted for duplicate)
> [INFO] |     +- org.apache.kafka:kafka_2.10:jar:0.8.2.2:compile
> [INFO] |     |  +- (org.apache.kafka:kafka-clients:jar:0.8.2.2:compile -
> omitted for conflict with 0.9.0.1)
> [INFO] |     |  +- com.yammer.metrics:metrics-core:jar:2.2.0:compile
> [INFO] |     |  |  \- (org.slf4j:slf4j-api:jar:1.7.2:compile - omitted for
> conflict with 1.7.7)
> [INFO] |     |  +- (org.scala-lang:scala-library:jar:2.10.4:compile -
> omitted for duplicate)
> [INFO] |     |  +- (org.apache.zookeeper:zookeeper:jar:3.4.6:compile -
> omitted for duplicate)
> [INFO] |     |  \- com.101tec:zkclient:jar:0.3:compile
> [INFO] |     |     \- (org.apache.zookeeper:zookeeper:jar:3.3.1:compile -
> omitted for conflict with 3.4.6)
> [INFO] |     +- (com.google.code.findbugs:jsr305:jar:1.3.9:compile - omitted
> for duplicate)
> [INFO] |     +- (org.apache.commons:commons-lang3:jar:3.3.2:compile -
> omitted for duplicate)
> [INFO] |     +- (org.slf4j:slf4j-api:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |     +- (org.slf4j:slf4j-log4j12:jar:1.7.7:compile - omitted for
> duplicate)
> [INFO] |     +- (log4j:log4j:jar:1.2.17:compile - omitted for duplicate)
> [INFO] |     \- (org.apache.flink:force-shading:jar:1.1.2:compile - omitted
> for duplicate)
> [INFO] \-
> org.apache.beam:beam-sdks-java-io-kafka:jar:0.3.0-incubating:compile
> [INFO]    +-
> (org.apache.beam:beam-sdks-java-core:jar:0.3.0-incubating:compile - omitted
> for duplicate)
> [INFO]    +- org.apache.kafka:kafka-clients:jar:0.9.0.1:compile
> [INFO]    |  +- (org.slf4j:slf4j-api:jar:1.7.6:compile - omitted for
> conflict with 1.7.7)
> [INFO]    |  +- (org.xerial.snappy:snappy-java:jar:1.1.1.7:compile - omitted
> for conflict with 1.1.2.1)
> [INFO]    |  \- net.jpountz.lz4:lz4:jar:1.2.0:compile
> [INFO]    +- org.slf4j:slf4j-api:jar:1.7.14:compile
> [INFO]    +- joda-time:joda-time:jar:2.4:compile
> [INFO]    +-
> com.fasterxml.jackson.core:jackson-annotations:jar:2.7.2:compile
> [INFO]    +- com.google.guava:guava:jar:19.0:compile
> [INFO]    \- com.google.code.findbugs:annotations:jar:3.0.1:compile
> [INFO]       +- net.jcip:jcip-annotations:jar:1.0:compile
> [INFO]       \- (com.google.code.findbugs:jsr305:jar:3.0.1:compile - omitted
> for conflict with 1.3.9)
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 2.945 s
> [INFO] Finished at: 2016-11-29T14:55:24-05:00
> [INFO] Final Memory: 18M/194M
> [INFO]
> ------------------------------------------------------------------------
>
> -------------------------------------
>
>
>
>
> On 2016-11-28 02:09 PM, Wayne Collins wrote:
>>
>> Hi Max,
>>
>> I pulled the binaries from the website but it looks like I pulled the
>> wrong version for 0.3.0 (Flink 1.1.3 instead of 1.1.2).
>> The message was the same for all combinations in testing my code but I
>> didn't try all the combinations with Emanuele's demo code in the sandbox.
>>
>> I'll rebuild my sandbox environment with just Beam 0.3.0 and Flink 1.1.2
>> and post the message and associated maven dependency tree.
>>
>> Thanks for your input!
>> Wayne
>>
>>
>> On 2016-11-28 01:35 PM, Maximilian Michels wrote:
>>>
>>> Hi Wayne,
>>>
>>> That seems like conflicting Guava versions in the classpath. Did you
>>> use Flink binaries from the website or did you compile Flink yourself?
>>> Is the error message the same across all Flink Beam combinations? If
>>> you use 0.2.0 the Flink version has to be 1.0.3. If you use 0.3.0, you
>>> will have to run it on Flink 1.1.2 because these are the Flink
>>> versions which the Beam releases are built upon. This will also be
>>> part of the upcoming documentation.
>>>
>>> For displaying conflicting versions, `mvn dependency:tree -Dverbose`
>>> provides a way to display conflicting versions of dependencies. It
>>> seems like Guava 18 and 19 are both used as dependencies which could
>>> result in a conflict (if shading is not set up properly). That's just
>>> a quick assessment but I would like to look further into the issue.
>>>
>>>
>>> -Max
>>>
>>>
>>> On Fri, Nov 25, 2016 at 4:48 PM, Wayne Collins <wayn...@dades.ca> wrote:
>>>>
>>>> Hi Stephen,
>>>>
>>>> Thanks for the suggestions!
>>>> I had come to the same conclusion but haven't been able to shade or set
>>>> "provides" to work around it.
>>>> The "mvn dependency:tree" looks good to me...
>>>>
>>>> Can anyone share a pom dependencies/exclusions fragment that is working
>>>> for
>>>> them with KafkaIO on a Flink/Yarn cluster?
>>>>
>>>> Thanks,
>>>> Wayne
>>>>
>>>>
>>>> Subject:
>>>> Re: java.lang.IllegalAccessError with KafkaIO
>>>> From:
>>>> Stephan Ewen <se...@apache.org>
>>>> Date:
>>>> 2016-11-24 02:38 PM
>>>> To:
>>>> user@beam.incubator.apache.org, d...@beam.incubator.apache.org
>>>>
>>>> I think have seen these kind of errors before.
>>>> It is the JVM checking upon class loading (lazy linking) that a
>>>> private/protected access is valid.
>>>>
>>>> It could be that this violation is because there are multiple versions
>>>> of
>>>> that class loaded - some in the SystemClassLoader and some in the user
>>>> library jar file.
>>>>
>>>> Fixing this probably needs some dependency cleanup, for example properly
>>>> setting dependencies as "provided" that should not be packaged again
>>>> into
>>>> the user code jar. Also, reducing visible dependencies by shading them
>>>> away
>>>> helps.
>>>>
>>>>
>>>> (snip)
>>
>>
>

Reply via email to