Thanks Thomas. Could you suggest a way so that I can figure out the path to particular localized file on container node preferably using OperatorContext ?
On Fri, Mar 17, 2017 at 8:06 PM, Thomas Weise <[email protected]> wrote: > If you use LIB_JARS or FILES, then those files will be localized by YARN > on the container node, you don't need to manually copy them from HDFS or > write cleanup code for it. > > Thomas > > On Fri, Mar 17, 2017 at 12:49 AM, vikram patil <[email protected]> > wrote: > >> Hello All, >> >> I am working on operator which would download the file from HDFS and >> store in on the local machine during run-time. For storing files to HDFS >> from client machine I will be using LIB_JARS or FILES configuration. Once >> operator fails/shuts down, I would like to clean up these files >> automatically if possible. >> >> Thanks & Regards, >> Vikram >> > >
