Owen O'Malley wrote:
Currently there isn't a way to do that. In Hadoop 0.19, there will be a way to have a clean up method that runs at the end of the job. See HADOOP-3150<https://issues.apache.org/jira/browse/HADOOP-3150> .
another bit of feature creep would be an expires: attribute on files, and something to purge expired files every so often. Which ensures that even if a job dies or the entire cluster is reset, stuff gets cleaned up
Before someone rushes to implement this, I've been burned in the past by differences in a clusters machines and clocks. Even if everything really is in sync with NTP, and not configured to talk to a NTP server that the production site can't see, you still need to be 100% that all your boxes are in the same time zone.
-steve
