[
https://issues.apache.org/jira/browse/HDFS-7607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated HDFS-7607:
------------------------------------------
Target Version/s: (was: 2.8.0)
Not much going on here for a long time, dropping from 2.8.0.
Not putting any target-version either anymore, let's target this depending on
when there is patch activity.
> Use random rack-local node for webhdfs opens to avoid OOM on DNs
> ----------------------------------------------------------------
>
> Key: HDFS-7607
> URL: https://issues.apache.org/jira/browse/HDFS-7607
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode, webhdfs
> Affects Versions: 2.0.0-alpha
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Priority: Critical
>
> Webhdfs currently redirects a client to the DN that physically has one of the
> replicas. Unlike the hdfs data streamer protocol which can easily handle
> hundreds or thousands of connections, jetty has poor performance under heavy
> load. Webhdfs clients can easily overwhelm the DNs and likely cause OOMs or
> excessive GC.
> The NN should redirect the client to a rack-local location to distribute the
> webhdfs load across multiple hosts. The rack can then use the lightweight
> streamer protocol.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
