Daryn Sharp created HDFS-7597:
---------------------------------
Summary: Clients seeking over webhdfs may crash the NN
Key: HDFS-7597
URL: https://issues.apache.org/jira/browse/HDFS-7597
Project: Hadoop HDFS
Issue Type: Improvement
Components: webhdfs
Affects Versions: 2.0.0-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Critical
Webhdfs seeks involve closing the current connection, and reissuing a new open
request with the new offset. The RPC layer caches connections so the DN keeps
a lingering connection open to the NN. Connection caching is in part based on
UGI. Although the client used the same token for the new offset request, the
UGI is different which forces the DN to open another unnecessary connection to
the NN.
A job that performs many seeks will easily crash the NN due to fd exhaustion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
