[
https://issues.apache.org/jira/browse/HDFS-13640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gang Xie updated HDFS-13640:
----------------------------
Affects Version/s: 3.0.2
Description:
The ShortCircuit read is disabled by HDFS-2757 due to the inconsistency of the
block states. With this limitation, some streaming/messaging application could
not benefit the performance improvement from SCR. In our streaming system whose
storage is HDFS, has around 90% read on the last blocks. So, it's necessary to
enable SCR on the last block, especially the app could ensure the read after
the flush.
After look into the original issue in HDFS-2757, it could only happen when the
read beyond the flush or the local datanode is kicked out from the pipeline.
But if there is the data and the visible length of the block covers the read
length, we could still read the data from the block, right?
I didn't get a completed solution here. Any suggestion could be helpful.
Component/s: hdfs-client
Summary: enable ShortCircuit Read on UC block (was: s)
> enable ShortCircuit Read on UC block
> ------------------------------------
>
> Key: HDFS-13640
> URL: https://issues.apache.org/jira/browse/HDFS-13640
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 3.0.2
> Reporter: Gang Xie
> Priority: Major
>
> The ShortCircuit read is disabled by HDFS-2757 due to the inconsistency of
> the block states. With this limitation, some streaming/messaging application
> could not benefit the performance improvement from SCR. In our streaming
> system whose storage is HDFS, has around 90% read on the last blocks. So,
> it's necessary to enable SCR on the last block, especially the app could
> ensure the read after the flush.
> After look into the original issue in HDFS-2757, it could only happen when
> the read beyond the flush or the local datanode is kicked out from the
> pipeline. But if there is the data and the visible length of the block covers
> the read length, we could still read the data from the block, right?
>
> I didn't get a completed solution here. Any suggestion could be helpful.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
