Hi Jarek, Your test case is running against which hadoop version. Regards, Mohammad
________________________________ From: Jarek Jarcec Cecho <[email protected]> To: [email protected] Sent: Friday, October 12, 2012 11:27 AM Subject: Oozie tests seems to be requiring umask 022 Hi Oozie developers, I've recently worked on OOZIE-1012 and I've noticed that on my box oozie tests seems to be requiring umask 022 (linux, Ubuntu 12.04), which actually seems to be implied by HDFS not by Oozie. If I run tests with umask 002 (my default), most of the tests will fail with following message: Cannot lock storage build/test/data/dfs/name1. The directory is already locked. I believe that this error message is just a consequence of following NullPointerException that appears in setUp method of affected test cases: java.lang.NullPointerException at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422) at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280) at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124) at org.apache.oozie.test.XTestCase.setUpEmbeddedHadoop(XTestCase.java:708) at org.apache.oozie.test.XTestCase.setUp(XTestCase.java:281) at org.apache.oozie.test.XFsTestCase.setUp(XFsTestCase.java:58) at org.apache.oozie.action.hadoop.ActionExecutorTestCase.setUp(ActionExecutorTestCase.java:62) I believe that reason of this NullPointerException is absence of usable data directories as *-output logs contains following fragments: 12/10/12 11:16:12 WARN datanode.DataNode: Invalid directory in dfs.data.dir: Incorrect permission for build/test/data/dfs/data/data1, expected: rwxr-xr-x, while actual: rwxrwxr-x 12/10/12 11:16:12 WARN datanode.DataNode: Invalid directory in dfs.data.dir: Incorrect permission for build/test/data/dfs/data/data2, expected: rwxr-xr-x, while actual: rwxrwxr-x 12/10/12 11:16:12 ERROR datanode.DataNode: All directories in dfs.data.dir are invalid. Please note that this output was generated with umask 002 and changing it to 022 will fix the issue. Does anyone else noticed this behaviour (failures) as well? If so then it might make sense to document this need on HowToContribute page [1] and maybe improve the XTestCase.setUpEmbeddedHadoop by catching NPE and informing developer that we're not able to bootstrap MiniDFSCluster? Jarcec Links: 1: http://incubator.apache.org/oozie/HowToContribute.html
