Hi, I'm running a job that completes in about a 90 seconds, but takes about 10-15 minutes to run cleanup. I'm looking for ways to affect or even monitor the cleanup time. I'd like even advice about whether this is more of a setup issue (like where I'm storing files, with Accumulo and Hadoop temporary and log files all writing to the same disk because our cluster is tiny) or a job issue (can I throw more reducers at it ? the brief description of the OutputCommitter says it uses available reducers for cleanup) or a programming issue (in that case I'd post a different question)
Basically, I want to know if the first way to go at this is by reconfiguring the cluster or if I should be programming my way out of this? Thanks. -- View this message in context: http://old.nabble.com/Ways-to-improve-job-cleanup-speed-tp33377374p33377374.html Sent from the Hadoop core-user mailing list archive at Nabble.com.
