[ https://issues.apache.org/jira/browse/HADOOP-2462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Allen Wittenauer resolved HADOOP-2462. -------------------------------------- Resolution: Incomplete Stale. > MiniMRCluster does not utilize multiple local directories in > "mapred.local.dir" > ------------------------------------------------------------------------------- > > Key: HADOOP-2462 > URL: https://issues.apache.org/jira/browse/HADOOP-2462 > Project: Hadoop Common > Issue Type: Bug > Components: test > Affects Versions: 0.15.0 > Reporter: Konstantin Shvachko > > My hadoop-site.xml specifies 4 local directories > {code} > <property> > <name>mapred.local.dir</name> > <value>${hadoop.tmp.dir}/mapred/local1, ${hadoop.tmp.dir}/mapred/local2, > ${hadoop.tmp.dir}/mapred/local3, > ${hadoop.tmp.dir}/mapred/local4</value> > </property> > {code} > and I am looking at MiniMRCluster.TaskTrackerRunner > There are several things here: > # localDirBase value is set to > {code} > "/tmp/h/mapred/local1, /tmp/h/mapred/local2, /tmp/h/mapred/local3, > /tmp/h/mapred/local4" > {code} > and I get a hierarchy of directories with commas and spaces in the names. > I think this was not designed to work with multiple dirs. > # Further down, all new directories are generated with the same name > {code} > File ttDir = new File(localDirBase, > Integer.toString(trackerId) + "_" + 0); > {code} > So in fact only one directory is created. I think the intension was to have i > instead of 0 > {code} > File ttDir = new File(localDirBase, > Integer.toString(trackerId) + "_" + i); > {code} > # On windows MiniMRCluster.TaskTrackerRunner in this case throws an > IOException, > which is silently ignored by all but the TestMiniMRMapRedDebugScript MiniMR > tests. > {code} > java.io.IOException: Mkdirs failed to create > /tmp/h/mapred/local1, /tmp/h/mapred/local2, /tmp/h/mapred/local3, > /tmp/h/mapred/local4/0_0 > at > org.apache.hadoop.mapred.MiniMRCluster$TaskTrackerRunner.<init>(MiniMRCluster.java:124) > at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:293) > at org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:244) > at > org.apache.hadoop.mapred.TestMiniMRClasspath.testClassPath(TestMiniMRClasspath.java:163) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:585) > at junit.framework.TestCase.runTest(TestCase.java:154) > at junit.framework.TestCase.runBare(TestCase.java:127) > at junit.framework.TestResult$1.protect(TestResult.java:106) > at junit.framework.TestResult.runProtected(TestResult.java:124) > at junit.framework.TestResult.run(TestResult.java:109) > at junit.framework.TestCase.run(TestCase.java:118) > at junit.framework.TestSuite.runTest(TestSuite.java:208) > at junit.framework.TestSuite.run(TestSuite.java:203) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:478) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:344) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196) > {code} > I am marking it as "Major" because we actually do not test multiple local > directories. > Looks like it was introduced rather recently by HADOOP-1819. -- This message was sent by Atlassian JIRA (v6.2#6252)