LocalDirAllocator should help with this. You can look through MapReduce code to see how it's used.
-Sandy On Mon, Jul 1, 2013 at 11:01 PM, Devaraj k <[email protected]> wrote: > You can make use of this configuration to do the same. **** > > ** ** > > <property>**** > > <description>List of directories to store *localized* files in. An *** > * > > application's *localized* file directory will be found in:**** > > ${yarn.nodemanager.local-*dirs*}/*usercache*/${user}/*appcache* > /application_${*appid*}.**** > > Individual containers' work directories, called container_${*contid*}, > will**** > > be *subdirectories* of this.**** > > </description>**** > > <name>yarn.nodemanager.local-*dirs*</name>**** > > <value>${hadoop.tmp.dir}/*nm*-local-*dir*</value>**** > > </property>**** > > ** ** > > Thanks**** > > Devaraj k**** > > ** ** > > *From:* John Lilley [mailto:[email protected]] > *Sent:* 02 July 2013 02:08 > *To:* [email protected] > *Subject:* temporary folders for YARN tasks**** > > ** ** > > When a YARN app and its tasks wants to write temporary files, how does it > know where to write the files? **** > > I am assuming that each task has some temporary space available, and I > hope it is available across multiple disk volumes for parallel performance. > **** > > Are those files cleaned up automatically after task exit?**** > > If I want to give lifetime control of the files to an auxiliary service > (along the lines of MR shuffle passing files to the aux service), how would > I do that, and would that entail different file locations?**** > > Thanks**** > > John**** > > ** ** > > ** ** >
