[
https://issues.apache.org/jira/browse/PIG-1838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989481#comment-12989481
]
Allen Wittenauer commented on PIG-1838:
---------------------------------------
I have a few thoughts on a better/programmatic way for Pig to be better behaved
without depending on users doing the right thing. One or more of these would
probably work:
a) Redefine java.io.tmpdir itself after it gets the Hadoop property files loaded
b) In the pig wrapper script, parse mapred-site.xml and pull out the mapred tmp
space
c) Override Java's createTempFile method to use Hadoop's tmp
location/$TEMPDIR/$TMPDIR/$TEMP/some other value
d) Change the jar assembly such that it goes into a
create->submit->delete->repeat pattern. (From a casual glance, it appears to
create all the jars at once rather than just when needed.)
> On a large farm, some pigs die of /tmp starvation
> -------------------------------------------------
>
> Key: PIG-1838
> URL: https://issues.apache.org/jira/browse/PIG-1838
> Project: Pig
> Issue Type: Wish
> Components: impl
> Affects Versions: 0.8.0
> Reporter: Allen Wittenauer
>
> We're starting to issues where interactive/command line pig users blow up due
> to so many large jar creations in /tmp. (In other words, pig execution prior
> to the java.io.tmpdir fix that Hadoop makes can kick in.) Pig should
> probably not depend upon users being savvy enough to override java.io.tmpdir
> on their own in these situations and/or a better steward of the space it does
> use.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
