[
https://issues.apache.org/jira/browse/HDFS-13564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka updated HDFS-13564:
---------------------------------
Fix Version/s: (was: 3.0.2)
> PreAllocator for DfsClientShm
> -----------------------------
>
> Key: HDFS-13564
> URL: https://issues.apache.org/jira/browse/HDFS-13564
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 3.0.2
> Reporter: Gang Xie
> Assignee: Lisheng Sun
> Priority: Minor
>
> When we do a stress test against Short-Circuit Local Reads, and found a
> bottleneck that allocating new DfsClientShm blocks a lot of slot allocatings
> on it.
> Currently, there are 128 slots per shm which means at most, 128 reads could
> be blocked by the shm allocation. Especially when stressed, the domain socket
> communication to datanode gets slow, and datanode could also have GC, which
> could cause some hundreds ms to allocate 1 shm, in turn, the reads. This is
> bad for some latency sensitive service, like Hbase.
> I'm working on the prototype and will upload the code and test result later.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
