daimin created HDFS-16520:
-----------------------------
Summary: Improve EC pread: avoid potential reading whole block
Key: HDFS-16520
URL: https://issues.apache.org/jira/browse/HDFS-16520
Project: Hadoop HDFS
Issue Type: Improvement
Components: dfsclient, ec
Affects Versions: 3.3.2, 3.3.1
Reporter: daimin
Assignee: daimin
HDFS client 'pread' represents 'position read', this kind of read just need a
range of data instead of reading the whole file/block. By using
BlockReaderFactory#setLength, client tells datanode the block length to be read
from disk and sent to client.
To EC file, the block length to read is not well set, by default using
'block.getBlockSize() - offsetInBlock' to both pread and sread. Thus datanode
read much more data and send to client, and abort when client closes
connection. There is a lot waste of resource to this situation.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
