We probably miss understand each other sir. I've experienced described issues during building Oozie on my local box before any deployment for real life tests. I've justed checkout repository and tried to build it (mvn package) - and almost all tests failed as I described in my original email. Therefore I believe that there is no other special log other than test related logs that I've read and provided appropriate lines, or at least those lines that I believe are related to the issue. I might send entire logs if you want me to.
Jarcec On Sun, Oct 14, 2012 at 03:06:16PM -0700, Mohammad Islam wrote: > You have to see the launcher log. > 1. First click on the your ooze job. > 2. Select the corresponding shell action. it will pop-up a new window. > 3. Select the right of "Console URL" which will take you to the hadoop JT > site. > 4. Select hadoop map task and click through to find the log. > > Regards, > Mohammad > > > ________________________________ > From: Jarek Jarcec Cecho <[email protected]> > To: [email protected] > Sent: Sunday, October 14, 2012 2:52 PM > Subject: Re: Oozie tests seems to be requiring umask 022 > > Hi Mohamad, > I've actually did not do anything special. I've just checked out svn > repository and executed "mvn test". Based on the maven output I'm expecting > that Oozie is compiling against 1.0.1 and 2.0.0-alpha, I'm not sure which > version is used in tests thought. > > Appropriate maven output that used: > [INFO] Apache Oozie Hadoop 1.0.1.oozie-3.4.0-SNAPSHOT .... SUCCESS [0.236s] > [INFO] Apache Oozie Hadoop Distcp 1.0.1.oozie-3.4.0-SNAPSHOT SUCCESS [0.151s] > [INFO] Apache Oozie Hadoop 1.0.1.oozie-3.4.0-SNAPSHOT Test SUCCESS [0.360s] > [INFO] Apache Oozie Hadoop 2.0.0-alpha.oozie-3.4.0-SNAPSHOT SUCCESS [0.427s] > [INFO] Apache Oozie Hadoop 2.0.0-alpha.oozie-3.4.0-SNAPSHOT Test SUCCESS > [0.411s] > [INFO] Apache Oozie Hadoop Distcp 2.0.0-alpha.oozie-3.4.0-SNAPSHOT SUCCESS > [0.137s] > > Jarcec > > On Fri, Oct 12, 2012 at 09:55:09PM -0700, Mohammad Islam wrote: > > Hi Jarek, > > Your test case is running against which hadoop version. > > Regards, > > Mohammad > > > > > > ________________________________ > > From: Jarek Jarcec Cecho <[email protected]> > > To: [email protected] > > Sent: Friday, October 12, 2012 11:27 AM > > Subject: Oozie tests seems to be requiring umask 022 > > > > Hi Oozie developers, > > I've recently worked on OOZIE-1012 and I've noticed that on my box oozie > > tests seems to be requiring umask 022 (linux, Ubuntu 12.04), which actually > > seems to be implied by HDFS not by Oozie. > > > > If I run tests with umask 002 (my default), most of the tests will fail > > with following message: > > > > Cannot lock storage build/test/data/dfs/name1. The directory is already > > locked. > > > > I believe that this error message is just a consequence of following > > NullPointerException that appears in setUp method of affected test cases: > > > > java.lang.NullPointerException > > at > > org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422) > > at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280) > > at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124) > > at org.apache.oozie.test.XTestCase.setUpEmbeddedHadoop(XTestCase.java:708) > > at org.apache.oozie.test.XTestCase.setUp(XTestCase.java:281) > > at org.apache.oozie.test.XFsTestCase.setUp(XFsTestCase.java:58) > > at > > org.apache.oozie.action.hadoop.ActionExecutorTestCase.setUp(ActionExecutorTestCase.java:62) > > > > I believe that reason of this NullPointerException is absence of usable > > data directories as *-output logs contains following fragments: > > > > 12/10/12 11:16:12 WARN datanode.DataNode: Invalid directory in > > dfs.data.dir: Incorrect permission for build/test/data/dfs/data/data1, > > expected: rwxr-xr-x, while actual: rwxrwxr-x > > 12/10/12 11:16:12 WARN datanode.DataNode: Invalid directory in > > dfs.data.dir: Incorrect permission for build/test/data/dfs/data/data2, > > expected: rwxr-xr-x, while actual: rwxrwxr-x > > 12/10/12 11:16:12 ERROR datanode.DataNode: All directories in dfs.data.dir > > are invalid. > > > > > > Please note that this output was generated with umask 002 and changing it > > to 022 will fix the issue. > > > > Does anyone else noticed this behaviour (failures) as well? If so then it > > might make sense to document this need on HowToContribute page [1] and > > maybe improve the XTestCase.setUpEmbeddedHadoop by catching NPE and > > informing developer that we're not able to bootstrap MiniDFSCluster? > > > > Jarcec > > > > Links: > > 1: http://incubator.apache.org/oozie/HowToContribute.html
signature.asc
Description: Digital signature
