[
https://issues.apache.org/jira/browse/HDFS-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14166924#comment-14166924
]
Daryn Sharp commented on HDFS-7191:
-----------------------------------
It's a slow writer and the maxIdleTime config is being exceeded. It's used to
set the socket's so timeout.
> WebHDFS prematurely closes connections under high concurrent loads
> ------------------------------------------------------------------
>
> Key: HDFS-7191
> URL: https://issues.apache.org/jira/browse/HDFS-7191
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Haohui Mai
> Priority: Critical
>
> We're seeing the DN prematurely closes APPEND connections:
> {noformat]
> 2014-09-22 23:53:12,721 WARN
> org.apache.hadoop.hdfs.web.resources.ExceptionHandler: INTERNAL_SERVER_ERROR
> java.nio.channels.CancelledKeyException
> at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:55)
> at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:59)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.updateKey(SelectChannelEndPoint.java:325)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.blockReadable(SelectChannelEndPoint.java:242)
> at
> org.mortbay.jetty.HttpParser$Input.blockForContent(HttpParser.java:1169)
> at org.mortbay.jetty.HttpParser$Input.read(HttpParser.java:1122)
> at java.io.InputStream.read(InputStream.java:85)
> at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:84)
> at
> org.apache.hadoop.hdfs.server.datanode.web.resources.DatanodeWebHdfsMethods.put(DatanodeWebHdfsMethods.java:239)
> at
> org.apache.hadoop.hdfs.server.datanode.web.resources.DatanodeWebHdfsMethods.access$000(DatanodeWebHdfsMethods.java:87)
> at
> org.apache.hadoop.hdfs.server.datanode.web.resources.DatanodeWebHdfsMethods$1.run(DatanodeWebHdfsMethods.java:205)
> at
> org.apache.hadoop.hdfs.server.datanode.web.resources.DatanodeWebHdfsMethods$1.run(DatanodeWebHdfsMethods.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at
> org.apache.hadoop.hdfs.server.datanode.web.resources.DatanodeWebHdfsMethods.put(DatanodeWebHdfsMethods.java:202)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
