Thanks for trying to fix it Ted. I think we need to carry our difficulty back up to our parent project brethren (Sounds like you have done this off-list; lets do it in jira/mail too). Your digging in to nail down the actual issue will help making our case. Maybe we can get/help the upstream crew make another release (I don't see a note on a new 2.0 release up on hadoop -- have I missed it)?
Thanks, St.Ack On Fri, Feb 22, 2013 at 11:49 AM, Ted Yu <[email protected]> wrote: > Trying out #2, I got: > > > testExcludeAllFromMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat) > Time elapsed: 35.746 sec <<< FAILURE! > java.lang.AssertionError > at org.junit.Assert.fail(Assert.java:86) > at org.junit.Assert.assertTrue(Assert.java:41) > at org.junit.Assert.assertTrue(Assert.java:52) > at > > org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468) > at > > org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testExcludeAllFromMinorCompaction(TestHFileOutputFormat.java:713) > > Arun Murthy got to know this defect. Another 2.0 release is coming soon > (next week, hopefully). > > I plan to wait for the new release. > > Thanks > > On Fri, Feb 22, 2013 at 11:39 AM, Ted Yu <[email protected]> wrote: > > > The root cause was the removal of capacity-scheduler.xml< > http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml> > from > > 2.0.3-alpha maven artifacts. > > This config file is needed to start CapacityScheduler which is the > default. > > > > After talking to Sid, I got two suggestions: > > 1. embed minimal capacity-scheduler.xml< > http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml> > in > > our src/test/resources > > 2. use FiFo scheduler instead which doesn't require presence of config > file > > > > I think #2, though temporary fix, is easy to change once a future 2.0 > > release addresses this issue. > > > > Once consensus is reached, I will attach another patch to HBASE-7904 > > > > Cheers > > > > > > On Fri, Feb 22, 2013 at 10:31 AM, Andrew Purtell <[email protected] > >wrote: > > > >> For minicluster unit tests? In my opinion, no. > >> > >> > >> On Fri, Feb 22, 2013 at 10:10 AM, Ted Yu <[email protected]> wrote: > >> > >> > I can try to find the proper usage. > >> > > >> > The question is: do we have to maintain compatibility with > 2.0.2-alpha ? > >> > > >> > Thanks > >> > > >> > On Fri, Feb 22, 2013 at 10:07 AM, Andrew Purtell <[email protected] > >> > >wrote: > >> > > >> > > Sigh. Every release seems to break our uses of MiniMRCluster all > over > >> > > again. > >> > > > >> > > > >> > > On Fri, Feb 22, 2013 at 10:04 AM, Ted Yu <[email protected]> > wrote: > >> > > > >> > > > I temporarily reverted this change due to: > >> > > > > >> > > > org.apache.hadoop.yarn.YarnException: Failed to Start > >> > > > org.apache.hadoop.mapred.MiniMRCluster > >> > > > at > >> > > > > >> > > > > >> > > > >> > > >> > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78) > >> > > > at > >> > > > > >> > > > > >> > > > >> > > >> > org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:75) > >> > > > at > >> > > > > >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:180) > >> > > > at > >> > > > > >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:170) > >> > > > at > >> > > > > >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:162) > >> > > > at > >> > > > > >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:123) > >> > > > at > >> > > > > >> > > > > >> > > > >> > > >> > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster(HBaseTestingUtility.java:1574) > >> > > > ... > >> > > > Caused by: org.apache.hadoop.yarn.YarnException: > >> > > > java.lang.IllegalStateException: Queue configuration missing child > >> > queue > >> > > > names for root > >> > > > at > >> > > > > >> > > > > >> > > > >> > > >> > org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.start(MiniYARNCluster.java:181) > >> > > > at > >> > > > > >> > > > > >> > > > >> > > >> > org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) > >> > > > ... 38 more > >> > > > Caused by: java.lang.IllegalStateException: Queue configuration > >> missing > >> > > > child queue names for root > >> > > > at > >> > > > > >> > > > > >> > > > >> > > >> > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.parseQueue(CapacityScheduler.java:328) > >> > > > at > >> > > > > >> > > > > >> > > > >> > > >> > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initializeQueues(CapacityScheduler.java:255) > >> > > > at > >> > > > > >> > > > > >> > > > >> > > >> > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:220) > >> > > > at > >> > > > > >> > > > > >> > > > >> > > >> > org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.init(ResourceManager.java:226) > >> > > > at > >> > > > > >> > > > > >> > > > >> > > >> > org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.start(MiniYARNCluster.java:161) > >> > > > > >> > > > Cheers > >> > > > > >> > > > On Fri, Feb 22, 2013 at 9:55 AM, Andrew Purtell < > >> [email protected]> > >> > > > wrote: > >> > > > > >> > > > > Have you tried running the mapreduce unit tests with this > version > >> > Ted? > >> > > > Just > >> > > > > curious. They have given us grief in the past. > >> > > > > > >> > > > > > >> > > > > On Fri, Feb 22, 2013 at 9:35 AM, Ted Yu <[email protected]> > >> wrote: > >> > > > > > >> > > > > > Hi, > >> > > > > > I have integrated the following JIRA to trunk: > >> > > > > > HBASE-7904 Upgrade hadoop 2.0 dependency to 2.0.3-alpha > >> > > > > > > >> > > > > > Earlier this week, Ramkrishna encountered some problem where > >> HBase > >> > > was > >> > > > > > built against 2.0.2-alpha but hadoop cluster deployment was > >> using > >> > > > > > 2.0.3-alpha > >> > > > > > > >> > > > > > In case you use HBase against hadoop 2.0, please note this > >> change. > >> > > > > > > >> > > > > > Cheers > >> > > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > -- > >> > > > > Best regards, > >> > > > > > >> > > > > - Andy > >> > > > > > >> > > > > Problems worthy of attack prove their worth by hitting back. - > >> Piet > >> > > Hein > >> > > > > (via Tom White) > >> > > > > > >> > > > > >> > > > >> > > > >> > > > >> > > -- > >> > > Best regards, > >> > > > >> > > - Andy > >> > > > >> > > Problems worthy of attack prove their worth by hitting back. - Piet > >> Hein > >> > > (via Tom White) > >> > > > >> > > >> > >> > >> > >> -- > >> Best regards, > >> > >> - Andy > >> > >> Problems worthy of attack prove their worth by hitting back. - Piet Hein > >> (via Tom White) > >> > > > > >
