Any ideas on how "attempt*" directories getting created directly under
"mapred.local.dir" ? Pointers to parts of the source code would help too.
Thanks,
-Rakesh
From: rkothari_...@hotmail.com
To: mapreduce-user@hadoop.apache.org
Subject: mapred.local.dir cleanup
Date: Tue, 18 Jan 2011 17:20:04 -0800
Hi,
I am seeing lots of leftover directories going back as far as 12 days in the
task trackers "mapred.local.dir". These directories are for "M/R task attempts".
How are these directories end up in "mapred.local.dir" as from my
understanding these directories should be in
"mapred.local.dir/taskTracker/jobcache/job-Id/" and should be cleaned up once
the job finishes (or after some interval) ? How can I enable automatic cleanup
of these directories ?
A big chunk of these leftover directories were created the same day/time when I
bounced my hadoop cluster.
Any pointers is highly appreciated.
Thanks,
-Rakesh
