I'm running into an inconsistent problem where with multiple reducers (and
extremely small amounts of data) I'll hit a situation where a Reducer task
attempt will hang at 33% until the JobTracker starts up a 2nd attempt. It does
not seem to be tied to any 1 machine, and when I change the number of reducers
to 1, it runs fine every time.
log4j:ERROR Failed to flush writer,
java.io.InterruptedIOException
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:260)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:202)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:272)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:276)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:122)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:212)
at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:58)
at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:316)
at org.apache.log4j.WriterAppender.append(WriterAppender.java:160)
at
org.apache.hadoop.mapred.TaskLogAppender.append(TaskLogAppender.java:58)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
at
org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
at org.apache.log4j.Category.callAppenders(Category.java:206)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at
org.apache.commons.logging.impl.Log4JLogger.info(Log4JLogger.java:199)
at
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler.freeHost(ShuffleScheduler.java:345)
at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:152)
Aaron Baff | Developer | Telescope, Inc.
email: [email protected]<mailto:[email protected]> | office: 424
270 2913 | www.telescope.tv<http://www.telescope.tv>
<http://twitter.com/Telescope_Inc>AMERICAN IDOL is back and better than ever
with a new team of judges on Season 10! Voting begins Tuesday, March 1st so be
sure to tune in and vote to make sure your favorite contestant makes it through
each week (check local listings for dates & times). More info at
www.americanidol.com<http://www.americanidol.com/>.
The information contained in this email is confidential and may be legally
privileged. It is intended solely for the addressee. Access to this email by
anyone else is unauthorized. If you are not the intended recipient, any
disclosure, copying, distribution or any action taken or omitted to be taken in
reliance on it, is prohibited and may be unlawful. Any views expressed in this
message are those of the individual and may not necessarily reflect the views
of Telescope Inc. or its associated companies.
