[
https://issues.apache.org/jira/browse/HDFS-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13848545#comment-13848545
]
stack commented on HDFS-5664:
-----------------------------
bq. why HBase use one stream always for one HFile in history？
Opening a new stream requires a trip to the namenode. Doing that for every
get/scan would up latencies significantly (and likely buckle the NN). We have
discussed in the past opening a new stream if a long-running scan is started --
if we recognize a scan as long-running -- so we can exploit any DN
pipelining.... We also recently changed from seek+read to pread if we
recognize we have more than one scanner going against the same set of files
because it improved our throughput.
bq. A single DFSInputStream is designed to be used by a single thread only.
Thanks Colin. Then we should remove all synchronization if single threaded
only?
Could we save on NN trips if we had added a 'clone' of DFSIS where'd create a
new one passing in an existing one; the new DFSIS would use the block info the
original had already obtained which would be enough to get the new DFSIS off
the ground w/o a trip to the NN?
Thanks
> try to relieve the BlockReaderLocal read() synchronized hotspot
> ---------------------------------------------------------------
>
> Key: HDFS-5664
> URL: https://issues.apache.org/jira/browse/HDFS-5664
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 3.0.0, 2.2.0
> Reporter: Liang Xie
> Assignee: Liang Xie
>
> Current the BlockReaderLocal's read has a synchronized modifier:
> {code}
> public synchronized int read(byte[] buf, int off, int len) throws IOException
> {
> {code}
> In a HBase physical read heavy cluster, we observed some hotspots from
> dfsclient path, the detail strace trace could be found from:
> https://issues.apache.org/jira/browse/HDFS-1605?focusedCommentId=13843241&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13843241
> I haven't looked into the detail yet, put some raw ideas here firstly:
> 1) replace synchronized with try lock with timeout pattern, so could
> fail-fast, 2) fallback to non-ssr mode if get a local reader lock failed.
> There're two suitable scenario at least to remove this hotspot:
> 1) Local physical read heavy, e.g. HBase block cache miss ratio is high
> 2) slow/bad disk.
> It would be helpful to achive a lower 99th percentile HBase read latency
> somehow.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
