limit concurrent connections(data serving thread) in one datanode
-----------------------------------------------------------------
Key: HADOOP-3779
URL: https://issues.apache.org/jira/browse/HADOOP-3779
Project: Hadoop Core
Issue Type: Improvement
Components: dfs
Affects Versions: 0.17.1
Reporter: LN
Priority: Minor
i'm here after HADOOP-2341 and HADOOP-2346, in my hbase env, many opening
mapfiles cause datanode OOME(stack memory), because 2000+ data serving threads
in datanode process.
although HADOOP-2346 has implements timeouts, it will be some situation many
connection created before the read timeout(default 6min) reach. like hbase
does, it open all files on regionserver startup.
limit concurrent connections(data serving thread) will make datanode more
stable. and i think it could be done in SocketIOWithTimeout$SelectorPool#select:
1. in SelectorPool#select, record all waiting SelectorInfo instances in a List
at the beginning, and remove it after 'Selector#select' done.
2. before real 'select', do a limitation check, if reached, close the first
selectorInfo.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
