If you use LIB_JARS or FILES, then those files will be localized by YARN on the container node, you don't need to manually copy them from HDFS or write cleanup code for it.
Thomas On Fri, Mar 17, 2017 at 12:49 AM, vikram patil <[email protected]> wrote: > Hello All, > > I am working on operator which would download the file from HDFS and store > in on the local machine during run-time. For storing files to HDFS from > client machine I will be using LIB_JARS or FILES configuration. Once > operator fails/shuts down, I would like to clean up these files > automatically if possible. > > Thanks & Regards, > Vikram >
