[
https://issues.apache.org/jira/browse/HDFS-11068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15652862#comment-15652862
]
Uma Maheswara Rao G edited comment on HDFS-11068 at 11/10/16 3:13 AM:
----------------------------------------------------------------------
# -
{code}
+ public Map<Long, List<BlockMovingInfo>> getBlocksToMoveStorages() {
+ Map<Long, List<BlockMovingInfo>> trackIdVsBlocks = new LinkedHashMap<>();
+ synchronized (storageMovementBlocks) {
+ if (storageMovementBlocks.isEmpty()) {
+ return trackIdVsBlocks;
+ }
+ trackIdVsBlocks.putAll(storageMovementBlocks);
+ storageMovementBlocks.keySet().removeAll(trackIdVsBlocks.keySet());
+ }
+ return trackIdVsBlocks;
}
{code}
Here what if one trackId/blockcollection contains many blocks to move? So, how
about just keep once trackID per heartbeat? (Later we may need to sub decid
them into small batches with in tracked itself if blocks are many(ex: a file
contains many blocks))
# -
{quote}
+ // TODO: Temporarily using the results from StoragePolicySatisfier
+ // class. This has to be revisited as part of HDFS-11029.
{quote}
HDFS-11029 almost ready. we can incorporate required changes and remove this
TODO. Thanks for adding TODO.
Other than this comments, patch looks great. Thanks
was (Author: umamaheswararao):
{code}
+ public Map<Long, List<BlockMovingInfo>> getBlocksToMoveStorages() {
+ Map<Long, List<BlockMovingInfo>> trackIdVsBlocks = new LinkedHashMap<>();
+ synchronized (storageMovementBlocks) {
+ if (storageMovementBlocks.isEmpty()) {
+ return trackIdVsBlocks;
+ }
+ trackIdVsBlocks.putAll(storageMovementBlocks);
+ storageMovementBlocks.keySet().removeAll(trackIdVsBlocks.keySet());
+ }
+ return trackIdVsBlocks;
}
{code}
Here what if one trackId/blockcollection contains many blocks to move? So, how
about just keep once trackID per heartbeat? (Later we may need to sub decid
them into small batches with in tracked itself if blocks are many(ex: a file
contains many blocks))
{quote}
+ // TODO: Temporarily using the results from StoragePolicySatisfier
+ // class. This has to be revisited as part of HDFS-11029.
{quote}
HDFS-11029 almost ready. we can incorporate required changes and remove this
TODO. Thanks for adding TODO.
Other than this comments, patch looks great. Thanks
> [SPS]: Provide unique trackID to track the block movement sends to coordinator
> ------------------------------------------------------------------------------
>
> Key: HDFS-11068
> URL: https://issues.apache.org/jira/browse/HDFS-11068
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode
> Affects Versions: HDFS-10285
> Reporter: Rakesh R
> Assignee: Rakesh R
> Attachments: HDFS-11068-HDFS-10285-01.patch,
> HDFS-11068-HDFS-10285.patch
>
>
> Presently DatanodeManager uses constant value -1 as
> [trackID|https://github.com/apache/hadoop/blob/HDFS-10285/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java#L1607],
> which is a temporary value. As per discussion with [~umamaheswararao], one
> proposal is to use {{BlockCollectionId/InodeFileId}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
