[
https://issues.apache.org/jira/browse/BLUR-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484903#comment-13484903
]
Aaron McCurry commented on BLUR-40:
-----------------------------------
I have run into an issue. Running the default setup in BlurClusterTest,
however changing it to 755 works as expected. I feel like we should come up
with a different solution (or enhance this one) so that devs like me don't have
to modify code to run the tests.
Starting DataNode 0 with dfs.data.dir:
/tmp/cluster/dfs/data/data1,/tmp/cluster/dfs/data/data2
ERROR 20121026_09:01:21:021_EDT [main] datanode.DataNode: All directories in
dfs.data.dir are invalid.
ERROR 20121026_09:01:21:021_EDT [main] datanode.DataNode: All directories in
dfs.data.dir are invalid.
ERROR 20121026_09:01:21:021_EDT [main] blur.MiniCluster: error opening file
system
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)
at org.apache.blur.MiniCluster.startDfs(MiniCluster.java:435)
at org.apache.blur.MiniCluster.startDfs(MiniCluster.java:420)
at org.apache.blur.MiniCluster.startDfs(MiniCluster.java:416)
at org.apache.blur.MiniCluster.startBlurCluster(MiniCluster.java:129)
at
org.apache.blur.thrift.BlurClusterTest.startCluster(BlurClusterTest.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
It appears to me that the MiniDfsCluster does not obey the
"dfs.datanode.data.dir.perm" permission, in MiniDfsCluster method
startDataNodes, this is how it creates those sub dirs:
for (int i = curDatanodesNum; i < curDatanodesNum+numDataNodes; i++) {
Configuration dnConf = new Configuration(conf);
if (manageDfsDirs) {
File dir1 = new File(data_dir, "data"+(2*i+1));
File dir2 = new File(data_dir, "data"+(2*i+2));
dir1.mkdirs();
dir2.mkdirs();
if (!dir1.isDirectory() || !dir2.isDirectory()) {
throw new IOException("Mkdirs failed to create directory for DataNode
"
+ i + ": " + dir1 + " or " + dir2);
}
dnConf.set(DataNode.DATA_DIR_KEY, dir1.getPath() + "," +
dir2.getPath());
}
No permissions are set, but they checked during startup. If no valid
directories are found the datanode object return as null, hence the NPE.
> Test cases failing in BlurClusterTest due to invalid directory permissions
> --------------------------------------------------------------------------
>
> Key: BLUR-40
> URL: https://issues.apache.org/jira/browse/BLUR-40
> Project: Apache Blur
> Issue Type: Bug
> Reporter: Gagan Deep Juneja
> Attachments: 0001-BLUR-ID-40-fix-failing-tests.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
