I would find it useful to have some sort of listener mechanism, where you could register an object to be notified of a job completion event and then respond to it accordingly.
Matt On 8/28/07, Stu Hood <[EMAIL PROTECTED]> wrote: > > > Does anyone have any ideas on this issue? > > Otherwise, if I were to write a patch to add this option for jobs to Hadoop, > would it be useful for anyone else? > > Thanks > Stu > > -----Original Message----- > From: Stu Hood <[EMAIL PROTECTED]> > Sent: Fri, August 24, 2007 9:43 am > To: [email protected] > Subject: Removing files after processing > > > Hello, > > Whats the best way to go about doing cleanup after MapReduce jobs? I'd like > to have the job delete its input files when it has finished successfully (but > preferably before it is marked as having finished: so I don't have to deal > with a race condition). > > Obviously, I don't want to have to track which files are being processed for > each job, since that data is stored anyway. Also, I'm using > JobClient.submitJob(), so I can't sit around and wait to do the cleanup > manually. > > Any suggestions? > > Thanks! > > Stu Hood > Webmail.us > "You manage your business. We'll manage your email."(r)
