[
https://issues.apache.org/jira/browse/HDFS-15255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076313#comment-17076313
]
Lisheng Sun commented on HDFS-15255:
------------------------------------
Thanx [~sodonnell] for patient review and good suggestions.
{quote}
Looking at the code, it seems this new change (sort by storage type) is not
compatible with "sort by load" (HDFS-14882). Is that correct? If both are set,
then the comparator here would return just the last one:
{quote}
If both are set, then the comparator here would return both.
Since in follow code, when call Collections.sort(list, comp), list has been
sorted.
{code:java}
if (readConsiderLoad) {
Comparator<DatanodeInfoWithStorage> comp =
Comparator.comparingInt(DatanodeInfo::getXceiverCount);
secondarySort = list -> Collections.sort(list, comp);
}
{code}
{quote}
Just a question - if network distance is always the same, and there is only 1
SSD storage and this new feature is enabled. Would the SSD storage be the only
one ever picked? Could like result in load problems on that host?
{quote}
i think this problem is a trade off. Rely on the client to want a faster read
latency or focus on DN's load.
i can put this new feature before the LOAD in HDFS-14882.
> Consider StorageType when DatanodeManager#sortLocatedBlock()
> ------------------------------------------------------------
>
> Key: HDFS-15255
> URL: https://issues.apache.org/jira/browse/HDFS-15255
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Lisheng Sun
> Assignee: Lisheng Sun
> Priority: Major
> Attachments: HDFS-15255.001.patch
>
>
> When only one replica of a block is SDD, the others are HDD.
> When the client reads the data, the current logic is that it considers the
> distance between the client and the dn. I think it should also consider the
> StorageType of the replica. Priority to return a replica of the specified
> StorageType
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
