[
https://issues.apache.org/jira/browse/HDFS-13564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lisheng Sun reassigned HDFS-13564:
----------------------------------
Assignee: Lisheng Sun
> PreAllocator for DfsClientShm
> -----------------------------
>
> Key: HDFS-13564
> URL: https://issues.apache.org/jira/browse/HDFS-13564
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 3.0.2
> Reporter: Gang Xie
> Assignee: Lisheng Sun
> Priority: Minor
> Fix For: 3.0.2
>
>
> When we do a stress test against Short-Circuit Local Reads, and found a
> bottleneck that allocating new DfsClientShm blocks a lot of slot allocatings
> on it.
> Currently, there are 128 slots per shm which means at most, 128 reads could
> be blocked by the shm allocation. Especially when stressed, the domain socket
> communication to datanode gets slow, and datanode could also have GC, which
> could cause some hundreds ms to allocate 1 shm, in turn, the reads. This is
> bad for some latency sensitive service, like Hbase.
> I'm working on the prototype and will upload the code and test result later.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
