[
https://issues.apache.org/jira/browse/YARN-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15952196#comment-15952196
]
Steve Loughran commented on YARN-6414:
--------------------------------------
* test wise, it shows that actually the JAR update process needs to have more
rigorous testing: the patch should go in for all the subprojects so yetus tests
them all. I'll keep an eye on future changes there
* circular dependencies are generally considered "bad form" and shouldn't have
happened. More specifically, the use of the acronym "DAG" in the maven
dependency graph declares that the graph is "acyclic". Pulling in hbase as a
dependency of hadoop is trying to do what spark/hive have achieved: created a
cycle. Is there anyway to unwind the cycle so that there is an ATSvb2 server
module independent of the others, which pulls in both. That way, it can take in
any shaded guava libs from Hadoop & whatever HBase needs, while also allowing
Hadoop to make progress on the plan to drop jersey 1 and so avoid version
issues there ( HADOOP-13332 )
BTW, Hadoop may export Guava 11 but it's coded to not use classes cut out of
later versions (e.g. guava dropping stopwatch (HADOOP-11032). Even so, the need
to up the base guava version is a serious need, not just for downestream
projects, but even because things like Curator which we pull in depends on
later versions. Every upgrade of curator has problems related to guava
HADOOP-11102 , HADOOP-11612
Side issue: In an ideal world. Guava would be backwards compatible, It isn't ,
and we get to deal with the pain. Whatever we do, something breaks. Oh, and
then there's protobuf.
> ATSv2 tests fail due to guava version upgrade
> ---------------------------------------------
>
> Key: YARN-6414
> URL: https://issues.apache.org/jira/browse/YARN-6414
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: 3.0.0-alpha3
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
> Reporter: Sonia Garudi
> Assignee: Haibo Chen
> Labels: ppc64le, x86_64
> Attachments: YARN-6414.00.patch, YARN-6414.01.patch
>
>
> Test failures seen in Hadoop YARN Timeline Service HBase tests project with
> following error :
> {code}
> java.lang.NoClassDefFoundError: com/google/common/io/LimitInputStream
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:223)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:913)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:899)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:722)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:660)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:279)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:955)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:700)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:529)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:585)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:751)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:735)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1407)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.createNameNode(MiniDFSCluster.java:998)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:869)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:704)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:642)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:590)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:987)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:868)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:862)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:806)
> at
> org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage.setup(TestTimelineReaderWebServicesHBaseStorage.java:88)
> {code}
> Below are the failing tests :
> {code}
> TestTimelineReaderWebServicesHBaseStorage.setup:88 » NoClassDefFound
> com/googl...
> TestHBaseTimelineStorageEntities.setupBeforeClass:100 » NoClassDefFound
> com/go...
> TestHBaseStorageFlowActivity.setupBeforeClass:72 » NoClassDefFound
> com/google/...
> TestHBaseStorageFlowRunCompaction.setupBeforeClass:84 » NoClassDefFound
> com/go...
> TestHBaseStorageFlowRun.setupBeforeClass:86 » NoClassDefFound
> com/google/commo...
> TestHBaseTimelineStorageApps.setupBeforeClass:93 » NoClassDefFound
> com/google/...
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
