Setting default tmp directory for java createTempFile (java.io.tmpdir)
----------------------------------------------------------------------
Key: HADOOP-2735
URL: https://issues.apache.org/jira/browse/HADOOP-2735
Project: Hadoop Core
Issue Type: New Feature
Components: mapred
Reporter: Koji Noguchi
Priority: Minor
On our cluster, we've seen Pig(http://incubator.apache.org/pig/) filling up the
/tmp and failing.
(also inefficient since all the local tasks were spilling to the same disk)
Pig is simply using java api createTempFile,
http://java.sun.com/j2se/1.5.0/docs/api/java/io/File.html#createTempFile(java.lang.String,%20java.lang.String,%20java.io.File
Can we add -Djava.io.tmpdir="./tmp" somewhere ?
so that,
1) Tasks can utilize all disks when using tmp
2) Any undeleted tmp files will be deleted by the tasktracker when task(job?)
is done.
The easiest way is to set it inside mapred.child.java.opts in the config, but
this can be overwritten if the users set their own task heapsize.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
