Bob Hansen created HDFS-9078: -------------------------------- Summary: WebHDFS does not support HTTP-keepalive Key: HDFS-9078 URL: https://issues.apache.org/jira/browse/HDFS-9078 Project: Hadoop HDFS Issue Type: Improvement Reporter: Bob Hansen
Since the inclusion of HDFS-7279, the netty back-end for WebHDFS is using HTTP 1.0, which does not support HDFS-keepalive. In the same use case as HDFS-8855, where a WebHDFS client is seeking around a file, this creates a tremendous amount of connections to the datanode, leaving tens of thousands of connections in a TIME_WAIT state. In some circumstances, all of the available ephemeral ports can be consumed with such connections and the datanode can fail. Configuring netty to support HTTP 1.1 and HTTP-keepalive would allow clients to re-use a single connection (much like HDFS-8855) and keep the number of used ports constrained. -- This message was sent by Atlassian JIRA (v6.3.4#6332)