The easiest way out would be to rename files to pick-up-able name upon successful copy, or have the loading done to a different directory and rename/move the file when successfully closed to the job input directory.
On Tue, May 1, 2012 at 3:22 PM, Stuti Awasthi <stutiawas...@hcl.com> wrote: > Hi All, > > > > I have a scenario in which Input files are copied to HDFS and MR jobs run on > the input directory. > > Now there can be a scenario in which file is getting copied to HDFS and MR > jobs starts , in this case I do not want my MR job to pick those files which > are getting copied to hdfs and process of copying is not complete. > > > > Is there any way/api to check that if the file is not completely written to > HDFS we can know. > > > > Regards, > > Stuti Awasthi > > HCL Comnet Systems and Services Ltd > > F-8/9 Basement, Sec-3,Noida. > > > > > ________________________________ > > > ::DISCLAIMER:: > ----------------------------------------------------------------------------------------------------------------------- > > The contents of this e-mail and any attachment(s) are confidential and > intended for the named recipient(s) only. > E-mail transmission cannot be guaranteed to be secure or error-free as > information could be intercepted, corrupted, > lost, destroyed, arrive late or incomplete, or contain viruses.The e mail > and its contents (with or without referred > errors) shall therefore not attach any liability on the originator or HCL or > its affiliates. Any views or opinions > presented in this email are solely those of the author and may not > necessarily reflect the opinions of HCL or its > affiliates. Any form of reproduction, dissemination, copying, disclosure, > Modification, distribution and/or publication > of this message without the prior written consent of the author of this > e-mail is strictly prohibited. If you have > received this email in error please delete it and notify the sender > immediately. Before opening any mail and attachments > please check them for viruses and defect. > > ----------------------------------------------------------------------------------------------------------------------- -- Harsh J
