Hi Wayne, That seems like conflicting Guava versions in the classpath. Did you use Flink binaries from the website or did you compile Flink yourself? Is the error message the same across all Flink Beam combinations? If you use 0.2.0 the Flink version has to be 1.0.3. If you use 0.3.0, you will have to run it on Flink 1.1.2 because these are the Flink versions which the Beam releases are built upon. This will also be part of the upcoming documentation.
For displaying conflicting versions, `mvn dependency:tree -Dverbose` provides a way to display conflicting versions of dependencies. It seems like Guava 18 and 19 are both used as dependencies which could result in a conflict (if shading is not set up properly). That's just a quick assessment but I would like to look further into the issue. -Max On Fri, Nov 25, 2016 at 4:48 PM, Wayne Collins <[email protected]> wrote: > Hi Stephen, > > Thanks for the suggestions! > I had come to the same conclusion but haven't been able to shade or set > "provides" to work around it. > The "mvn dependency:tree" looks good to me... > > Can anyone share a pom dependencies/exclusions fragment that is working for > them with KafkaIO on a Flink/Yarn cluster? > > Thanks, > Wayne > > > Subject: > Re: java.lang.IllegalAccessError with KafkaIO > From: > Stephan Ewen <[email protected]> > Date: > 2016-11-24 02:38 PM > To: > [email protected], [email protected] > > I think have seen these kind of errors before. > It is the JVM checking upon class loading (lazy linking) that a > private/protected access is valid. > > It could be that this violation is because there are multiple versions of > that class loaded - some in the SystemClassLoader and some in the user > library jar file. > > Fixing this probably needs some dependency cleanup, for example properly > setting dependencies as "provided" that should not be packaged again into > the user code jar. Also, reducing visible dependencies by shading them away > helps. > > > (snip)
