[
https://issues.apache.org/jira/browse/HADOOP-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12487062
]
Devaraj Das commented on HADOOP-1176:
-------------------------------------
I know the patch is preliminary. I just wanted to point out that we should use
standard HTTP headers as opposed to Jetty's constants. We should not rely on
Jetty's constants as far as possible especially when we are using the standard
HTTP.header strings.
> Reduce hang on huge map output
> ------------------------------
>
> Key: HADOOP-1176
> URL: https://issues.apache.org/jira/browse/HADOOP-1176
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.12.2
> Reporter: Hairong Kuang
> Assigned To: Arun C Murthy
> Fix For: 0.13.0
>
> Attachments: HADOOP-1176_20070405_1.patch
>
>
> I ran a job with one map and one reduce. The map succeeded, producing 3G
> output. The reducer hang on fetching the map output. The log shows the
> following error message:
> WARN org.apache.hadoop.mapred.TaskTracker:
> getMapOutput(task_0401_m_000000_0,0) failed :
> java.net.SocketException: Connection reset
> at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:96)
> at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
> at
> org.mortbay.http.ChunkingOutputStream.bypassWrite(ChunkingOutputStream.java:151)
> at
> org.mortbay.http.BufferedOutputStream.write(BufferedOutputStream.java:139)
> at org.mortbay.http.HttpOutputStream.write(HttpOutputStream.java:423)
> at org.mortbay.jetty.servlet.ServletOut.write(ServletOut.java:54)
> at
> org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:1664)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:427)
> at
> org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:475)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:567)
> at org.mortbay.http.HttpContext.handle(HttpContext.java:1565)
> at
> org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:635)
> at org.mortbay.http.HttpContext.handle(HttpContext.java:1517)
> at org.mortbay.http.HttpServer.service(HttpServer.java:954)
> at org.mortbay.http.HttpConnection.service(HttpConnection.java:814)
> at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:981)
> at org.mortbay.http.HttpConnection.handle(HttpConnection.java:831)
> at
> org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244)
> at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357)
> at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534)
> WARN /: /mapOutput?map=task_0401_m_000000_0&reduce=0:
> java.lang.IllegalStateException: Committed
> at
> org.mortbay.jetty.servlet.ServletHttpResponse.resetBuffer(ServletHttpResponse.java:212)
> at
> org.mortbay.jetty.servlet.ServletHttpResponse.sendError(ServletHttpResponse.java:375)
> at
> org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:1694)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
> at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:427)
> at
> org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:475)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:567)
> at org.mortbay.http.HttpContext.handle(HttpContext.java:1565)
> at
> org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:635)
> at org.mortbay.http.HttpContext.handle(HttpContext.java:1517)
> at org.mortbay.http.HttpServer.service(HttpServer.java:954)
> at org.mortbay.http.HttpConnection.service(HttpConnection.java:814)
> at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:981)
> at org.mortbay.http.HttpConnection.handle(HttpConnection.java:831)
> at
> org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244)
> at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357)
> at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
