I run unit test against 2.0.4-alpha in the trunk. The tests hit "OutOfMemoryError: PermGen space" in the middle, but so far no failures or errors. I re-kicked off the test.
Johnny On Fri, May 10, 2013 at 6:49 PM, Johnny Zhang <xiao...@cloudera.com> wrote: > Hi, Andrew: > I just set up a job to run unit test against 2.0.4-alpha. I will > investigate failure and reply to thread. > > Thanks, > Johnny Zhang > > > On Fri, May 10, 2013 at 6:19 PM, Andrew Purtell <apurt...@apache.org>wrote: > >> I've tried that, thanks. I did a bit more investigation and it seems the >> issue is recent Hadoop 2 releases. Has anyone tried running Pig unit tests >> using a more recent Hadoop release than 2.0.0-alpha? Maybe my trouble is a >> simple thing that someone with more experience with Pig internals would >> see >> right away? Cluster testing seems ok. It's just unit tests that fail. But >> that is concerning. >> >> I'm trying HEAD of branch-0.11. >> >> My Java is version "1.6.0_43" Java(TM) SE Runtime Environment (build >> 1.6.0_43-b01) Java HotSpot(TM) 64-Bit Server VM (build 20.14-b01, mixed >> mode). OS is Ubuntu 13.04 (GNU/Linux 3.8.0-19-generic x86_64). >> >> With defaults and only -Dhadoopversion=23 on the Ant command line, it >> seems >> ok. >> >> With build.properties of: >> >> hadoopversion=23 >> hadoop-common.version=2.0.4-alpha >> hadoop-hdfs.version=2.0.4-alpha >> hadoop-mapreduce.version=2.0.4-alpha >> >> >> or defined on the Ant command line, I'll see unit test failures like: >> >> Testcase: testAccumWithDistinct took 0.868 sec >> Caused an ERROR >> org/apache/hadoop/mapred/ResourceMgrDelegate >> java.lang.NoClassDefFoundError: >> org/apache/hadoop/mapred/ResourceMgrDelegate >> at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:112) >> at >> >> org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34) >> at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:94) >> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:81) >> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:74) >> at org.apache.hadoop.mapred.JobClient.init(JobClient.java:482) >> at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:461) >> at >> >> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:152) >> at org.apache.pig.PigServer.launchPlan(PigServer.java:1264) >> at >> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249) >> at org.apache.pig.PigServer.storeEx(PigServer.java:931) >> at org.apache.pig.PigServer.store(PigServer.java:898) >> at org.apache.pig.PigServer.openIterator(PigServer.java:811) >> at >> >> org.apache.pig.test.TestAccumulator.testAccumWithDistinct(TestAccumulator.java:424) >> >> That suggests a cause but I've not started spelunking code with the hope >> this is something simple that someone has already encountered. >> >> >> On Sat, May 11, 2013 at 1:31 AM, Johnny Zhang <xiao...@cloudera.com> >> wrote: >> >> > Hi, Andrew: >> > Does something like "-Dhadoopversion=23" help ? eg. ant clean test >> > -Dhadoopversion=23 -Dtest.junit.output.format=xml >> > >> > Johnny >> > >> > >> > On Fri, May 10, 2013 at 3:39 AM, Andrew Purtell <apurt...@apache.org> >> > wrote: >> > >> > > Please pardon the basic question. I'm building Pig 0.11.2-SNAPSHOT >> > against >> > > Hadoop 2.0.4. 'ant package' and full cluster tests work fine, but I'm >> not >> > > having much luck with running the unit tests, 'ant test-core' or 'ant >> > > test'. The problem looks to be a MR app classpath issue. >> > > >> > > Sometimes: java.lang.NoClassDefFoundError: >> > > org/apache/hadoop/yarn/client/YarnClientImpl >> > > >> > > Sometimes: java.lang.NoClassDefFoundError: >> > > org/apache/hadoop/mapred/ResourceMgrDelegate >> > > >> > > A few Google searches have turned up no useful pointers. Maybe there >> is >> > > something simple I am missing? How do you set up for running unit >> tests >> > on >> > > your dev boxes? >> > > >> > > -- >> > > Best regards, >> > > >> > > - Andy >> > > >> > > Problems worthy of attack prove their worth by hitting back. - Piet >> Hein >> > > (via Tom White) >> > > >> > >> >> >> >> -- >> Best regards, >> >> - Andy >> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein >> (via Tom White) >> > >
