Chris, This is what I see by running a mvn dependency:tree on hadoop. Both seem to be pulling in guava-11. This is a little strange since curator changed to 16 in March 2014, and before that were on 14.
Hadoop-2.6 [INFO] +- org.apache.curator:curator-framework:jar:2.6.0:compile [INFO] | +- org.apache.curator:curator-client:jar:2.6.0:compile [INFO] | \- com.google.guava:guava:jar:11.0.2:compile [INFO] | \- com.google.code.findbugs:jsr305:jar:1.3.9:compile branch-2 [INFO] +- org.apache.curator:curator-framework:jar:2.7.1:compile [INFO] | +- org.apache.curator:curator-client:jar:2.7.1:compile [INFO] | \- com.google.guava:guava:jar:11.0.2:compile [INFO] | \- com.google.code.findbugs:jsr305:jar:3.0.0:compile (version managed from 1.3.9) I'm guessing that maven does something here - either excluding the curator guava dependency from being pulled in by Hadoop, or it's fixing versions. The dist directory for Hadoop contains only the guava-11 jar. The same applies to Tez. I don't think auth will be using any dependencies from 16 given what the maven dependency tree gives. There's been a fair amount of discussion on Hadoop jiras to update the Guava version - I'm not sure what the resolution is though. >From the way Hadoop gets only guava11 in it's classpath, I'd say it's safe to remove the Guava16 jar in your setup. Is first in classpath a valid assumption ? It does work - but I'm not sure there's any guarantees / contracts for this to actually work. On a related note, Tez may end up shading the guava version it uses - and remove it from dependencies that are exported. On Thu, Feb 26, 2015 at 10:59 AM, Chris K Wensel <[email protected]> wrote: > I’m running Tez 0.6.0 with the default dependency of Hadoop 2.6 and am > receiving a fair number of these > > Exception in thread > "org.apache.hadoop.util.JvmPauseMonitor$Monitor@74173125" > java.lang.NoSuchMethodError: > com.google.common.base.Stopwatch.elapsedMillis()J > at > org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:184) > at java.lang.Thread.run(Thread.java:722) > > of which result in a complete failure to run Tez > > This is due to Tez/Yarn building against Guava 11.0.2, > > org.apache.tez:tez-api:0.6.0 > | +--- log4j:log4j:1.2.17 > | +--- com.google.guava:guava:11.0.2 -> 16.0.1 > > but Curator requiring 16.0.1 and forcing the up version > > org.apache.curator:curator-framework:2.6.0 > | | | | | +--- org.apache.curator:curator-client:2.6.0 > | | | | | | +--- org.slf4j:slf4j-api:1.7.6 > | | | | | | +--- org.apache.zookeeper:zookeeper:3.4.6 (*) > | | | | | | \--- com.google.guava:guava:16.0.1 > > this makes me wonder how Tez 0.6.0 passes any tests (a side effect of > maven?), let alone actually run on a cluster as curator is used by auth > which is used by common > > or is it a happy coincidence that auth doesn’t use any guava api’s not > available in 11.0.2? thus it’s safe to force a downgrade (or put 11.0.2 > first in the classpath). > > ckw > > — > Chris K Wensel > [email protected] > > > > >
