[
https://issues.apache.org/jira/browse/HDFS-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15465042#comment-15465042
]
Kai Zheng edited comment on HDFS-10794 at 9/6/16 7:44 AM:
----------------------------------------------------------
Thanks [~rakeshr] for the work. It's a good start.
1. Ref. the codes below
{code}
+ private final List<Future<Void>> moverTaskFuturesMap;
{code}
How about: moverTaskFuturesMap => moverTaskFutures
2. Maybe: {{BlockToMoveStorageTask}} => {{BlockMovingTask}};
{{BlockStorageMovementCommand}} => {{BlockMovingCommand}};
{{BlockInfoToMoveStorage}} => {{BlockMovingInfo}}?
3. In {{testMoveSingleBlockToAnotherDatanode}}, not sure about why it needs to
stop some datanode since what's to test is if storage policy is changed,
whether any block movement will happen.
4. Ref. the codes below,
{code}
+ StoragePolicySatisfyWorker worker = new StoragePolicySatisfyWorker(conf,
+ src);
+ List<BlockInfoToMoveStorage> blockToMoveStorageTasks = new ArrayList<>();
+ BlockInfoToMoveStorage storagePair = prepareBlockToMoveStoragePair(
+ lb.getBlock(), lb.getLocations()[0], targetDnInfo,
+ lb.getStorageTypes()[0], StorageType.ARCHIVE);
+ blockToMoveStorageTasks.add(storagePair);
+ INode inode = cluster.getNamesystem().getFSDirectory().getINode(file);
+ worker.processBlockToMoveStorageTasks(inode.getId(),
+ blockToMoveStorageTasks);
+ cluster.triggerHeartbeats();
{code}
Such codes are better to be equipped with some comments like: "TODO: it doesn't
need here and will be cleaned, when NN is implemented to be able to send block
moving commands."
was (Author: drankye):
Thanks [~Rakesh] for the work. It's a good start.
1. Ref. the codes below
{code}
+ private final List<Future<Void>> moverTaskFuturesMap;
{code}
How about: moverTaskFuturesMap => moverTaskFutures
2. Maybe: {{BlockToMoveStorageTask}} => {{BlockMovingTask}};
{{BlockStorageMovementCommand}} => {{BlockMovingCommand}};
{{BlockInfoToMoveStorage}} => {{BlockMovingInfo}}?
3. In {{testMoveSingleBlockToAnotherDatanode}}, not sure about why it needs to
stop some datanode since what's to test is if storage policy is changed,
whether any block movement will happen.
4. Ref. the codes below,
{code}
+ StoragePolicySatisfyWorker worker = new StoragePolicySatisfyWorker(conf,
+ src);
+ List<BlockInfoToMoveStorage> blockToMoveStorageTasks = new ArrayList<>();
+ BlockInfoToMoveStorage storagePair = prepareBlockToMoveStoragePair(
+ lb.getBlock(), lb.getLocations()[0], targetDnInfo,
+ lb.getStorageTypes()[0], StorageType.ARCHIVE);
+ blockToMoveStorageTasks.add(storagePair);
+ INode inode = cluster.getNamesystem().getFSDirectory().getINode(file);
+ worker.processBlockToMoveStorageTasks(inode.getId(),
+ blockToMoveStorageTasks);
+ cluster.triggerHeartbeats();
{code}
Such codes are better to be equipped with some comments like: "TODO: it doesn't
need here and will be cleaned, when NN is implemented to be able to send block
moving commands."
> [SPS]: Provide storage policy satisfy worker at DN for co-ordinating the
> block storage movement work
> ----------------------------------------------------------------------------------------------------
>
> Key: HDFS-10794
> URL: https://issues.apache.org/jira/browse/HDFS-10794
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode
> Affects Versions: HDFS-10285
> Reporter: Rakesh R
> Assignee: Rakesh R
> Attachments: HDFS-10794-00.patch, HDFS-10794-HDFS-10285.00.patch,
> HDFS-10794-HDFS-10285.01.patch, HDFS-10794-HDFS-10285.02.patch
>
>
> The idea of this jira is to implement a mechanism to move the blocks to the
> given target in order to satisfy the block storage policy. Datanode receives
> {{blocktomove}} details via heart beat response from NN. More specifically,
> its a datanode side extension to handle the block storage movement commands.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
