Hi Max,
I pulled the binaries from the website but it looks like I pulled the
wrong version for 0.3.0 (Flink 1.1.3 instead of 1.1.2).
The message was the same for all combinations in testing my code but I
didn't try all the combinations with Emanuele's demo code in the sandbox.
I'll rebuild my sandbox environment with just Beam 0.3.0 and Flink 1.1.2
and post the message and associated maven dependency tree.
Thanks for your input!
Wayne
On 2016-11-28 01:35 PM, Maximilian Michels wrote:
Hi Wayne,
That seems like conflicting Guava versions in the classpath. Did you
use Flink binaries from the website or did you compile Flink yourself?
Is the error message the same across all Flink Beam combinations? If
you use 0.2.0 the Flink version has to be 1.0.3. If you use 0.3.0, you
will have to run it on Flink 1.1.2 because these are the Flink
versions which the Beam releases are built upon. This will also be
part of the upcoming documentation.
For displaying conflicting versions, `mvn dependency:tree -Dverbose`
provides a way to display conflicting versions of dependencies. It
seems like Guava 18 and 19 are both used as dependencies which could
result in a conflict (if shading is not set up properly). That's just
a quick assessment but I would like to look further into the issue.
-Max
On Fri, Nov 25, 2016 at 4:48 PM, Wayne Collins <[email protected]> wrote:
Hi Stephen,
Thanks for the suggestions!
I had come to the same conclusion but haven't been able to shade or set
"provides" to work around it.
The "mvn dependency:tree" looks good to me...
Can anyone share a pom dependencies/exclusions fragment that is working for
them with KafkaIO on a Flink/Yarn cluster?
Thanks,
Wayne
Subject:
Re: java.lang.IllegalAccessError with KafkaIO
From:
Stephan Ewen <[email protected]>
Date:
2016-11-24 02:38 PM
To:
[email protected], [email protected]
I think have seen these kind of errors before.
It is the JVM checking upon class loading (lazy linking) that a
private/protected access is valid.
It could be that this violation is because there are multiple versions of
that class loaded - some in the SystemClassLoader and some in the user
library jar file.
Fixing this probably needs some dependency cleanup, for example properly
setting dependencies as "provided" that should not be packaged again into
the user code jar. Also, reducing visible dependencies by shading them away
helps.
(snip)
