[
https://issues.apache.org/jira/browse/HDFS-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427968#comment-16427968
]
Ajay Kumar commented on HDFS-9666:
----------------------------------
[~yangjiandan] thanks for updating the patch. Overall it looks good. Few
comments:
* DFSInputStream
** Personally i think we should merge getBestNodeDNAddrPair and
getBestNodeDNAddrPairRemoteSsdFirst as most of the functionality is same.
** Improve chooseDataNode javadoc for remoteSsdFirst (L860) to something like
"if true read remote SSD/RAM replica first if local Disks are HDD"
** Improve javadoc for getBestNodeDNAddrPairRemoteSsdFirst to mention the
selection strategy.
"Read from local node if a) If block is on SSD. b) If no other replica exist on
SSD or RAM
Read from remote node if local node replica is on HDD and remote node replica
is on SSD/RAM"
* TestDFSInputStream#testReadSsdFirstWithSsd
** Typo in L222 & L259. We should include expected storageId in Assert text.
i.e "Should be storageID3." & "Should be storageID1." respectively.
** Also we should assert expected storage type in both tests.
> Enable hdfs-client to read even remote SSD/RAM prior to local disk replica to
> improve random read
> -------------------------------------------------------------------------------------------------
>
> Key: HDFS-9666
> URL: https://issues.apache.org/jira/browse/HDFS-9666
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 2.6.0, 2.7.0
> Reporter: ade
> Assignee: Jiandan Yang
> Priority: Major
> Attachments: HDFS-9666.0.patch, HDFS-9666.001.patch,
> HDFS-9666.002.patch, HDFS-9666.003.patch, HDFS-9666.004.patch
>
>
> We want to improve random read performance of HDFS for HBase, so enabled the
> heterogeneous storage in our cluster. But there are only ~50% of datanode &
> regionserver hosts with SSD. we can set hfile with only ONE_SSD not ALL_SSD
> storagepolicy and the regionserver on none-SSD host can only read the local
> disk replica . So we developed this feature in hdfs client to read even
> remote SSD/RAM prior to local disk replica.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
