Roman, Is there a chance you can run the tests with the full stack built against branch-2.1-beta and help us know where we are?
thanks, Arun On Jun 16, 2013, at 4:50 PM, Arun C Murthy wrote: > Responses inline: > > On Jun 16, 2013, at 1:04 PM, Roman Shaposhnik wrote: > >> But there's a bit of bad news too (or at least the news that need to >> be triaged). At >> this point I don't know whether the Hadoop code is to blame or the >> tests/components >> themselves -- all I know is that these tests passed with Hadoop 2.0.5-alpha: >> 1. HDFS append integration tests failed: >> >> http://bigtop01.cloudera.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-runsmokes/lastCompletedBuild/testReport/org.apache.bigtop.itest.hadoop.hdfs/TestFileAppend/testMultipleOutputStreamFailure/ >> It seems that there's some incompatibility between the client code >> that was compiled against Hadoop 2.0.5 (as part of Bigtop 0.6.0 >> release) >> and the current Hadoop 2.1.0. > > This is well known, we need to recompile against hadoop-2.1.0-beta. > >> >> 2. Quite a few Sqoop tests ended up failing because of what seems >> like AM not >> realizing that one of the tasks exited and waiting for it to >> timeout. In the end >> the task is getting killed like this: >> AttemptID:attempt_1371348647940_0030_m_000000_2 Timed out >> after 600 secsContainer killed by the ApplicationMaster. >> but it takes a VERY long time (on the task side the log is >> attached bellow). > > Essentially, this is a Sqoop error - we could investigate why it took 600s, > but doesn't look like a blocker to me. > > We'll probably need another 2.1.1-beta anyway... > >> >> 3. There's a couple of Hive tests (out of more than a dozen) that >> failed in a pretty odd way >> (scroll to the very bottom of every page to see the excpetion): >> >> http://bigtop01.cloudera.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-runsmokes/59/testReport/org.apache.bigtop.itest.hivesmoke/TestHiveSmokeBulk/testHiveBulk_auto_join20_/ >> >> http://bigtop01.cloudera.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-runsmokes/59/testReport/org.apache.bigtop.itest.hivesmoke/TestHiveSmokeBulk/testHiveBulk_union3_/ >> What's peculiar here is that nothing has failed *before* or >> *after* these particular >> tests. Hence I don't think that the state of the cluster >> deployment is to blame. > > Both errors seemed to be related to Hive unit tests failing since > MiniHDFSCluster didn't come up: > > Job Submission failed with exception > 'org.apache.hadoop.ipc.RemoteException(File > /user/jenkins/.staging/job_1371348647940_0308/job.split could only be > replicated to 0 nodes instead of minReplication (=1). There are 4 > datanode(s) running and no node(s) are excluded in this operation. > > >> >> 4. All of the Mahout tests failed with the following: >> >> http://bigtop01.cloudera.org:8080/view/Bigtop-trunk/job/Bigtop-trunk-runsmokes/59/testReport/org.apache.bigtop.itest.mahout.smoke/TestMahoutExamples/clusterControlDataWithCanopy/ >> This odd, and as I said -- all I know at this point is that >> the very same >> tests running the very same Mahout pass with Hadoop 2.0.5-alpha. > > Again, this is related to fact that we need to recompile Mahout against > 2.1.0-beta - in particular this was due to the compatibility work done via > MAPREDUCE-5156 (part of MAPREDUCE-5108). > > > Arun > -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
