[
https://issues.apache.org/jira/browse/HDFS-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13847906#comment-13847906
]
Colin Patrick McCabe commented on HDFS-5664:
--------------------------------------------
A single {{DFSInputStream}} is designed to be used by a single thread only. We
don't allow multiple reads from multiple threads on the same stream to go
forward at the same time. So I don't see how making {{BlockReader}} (or a
subclass like {{BlockReaderLocal}}) concurrent would help at all, since there
would still be a big "synchronized" on all the {{DFSInputStream#read}} methods
which use the {{BlockReader}}.
If multiple threads want to read the same file at the same time, they can open
multiple distinct streams for it. At that point, they're not sharing the same
{{BlockReader}}, so whether or not BRL is synchronized doesn't matter.
> try to relieve the BlockReaderLocal read() synchronized hotspot
> ---------------------------------------------------------------
>
> Key: HDFS-5664
> URL: https://issues.apache.org/jira/browse/HDFS-5664
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 3.0.0, 2.2.0
> Reporter: Liang Xie
> Assignee: Liang Xie
>
> Current the BlockReaderLocal's read has a synchronized modifier:
> {code}
> public synchronized int read(byte[] buf, int off, int len) throws IOException
> {
> {code}
> In a HBase physical read heavy cluster, we observed some hotspots from
> dfsclient path, the detail strace trace could be found from:
> https://issues.apache.org/jira/browse/HDFS-1605?focusedCommentId=13843241&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13843241
> I haven't looked into the detail yet, put some raw ideas here firstly:
> 1) replace synchronized with try lock with timeout pattern, so could
> fail-fast, 2) fallback to non-ssr mode if get a local reader lock failed.
> There're two suitable scenario at least to remove this hotspot:
> 1) Local physical read heavy, e.g. HBase block cache miss ratio is high
> 2) slow/bad disk.
> It would be helpful to achive a lower 99th percentile HBase read latency
> somehow.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
