[
https://issues.apache.org/jira/browse/HDFS-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17296326#comment-17296326
]
Ayush Saxena commented on HDFS-15022:
-------------------------------------
I didn't check the code but the pdf attached and I actually didn't catch this
design and use case, the datanode goes down, and it was using a bucket and now
you want that bucket to be used by another datanode? Is this you want to
achieve? If so that you can do straight away? Add it to a DN and trigger Block
Report.
Or is it like you want to copy the blocks from this bucket to another buckets
directly? So how do you prevent duplicate replicas on one DN, means if you land
up copying a block to a DN which already has that.
If all stuffs are sorted then regarding the new API, can't we use the same
interface as Mover/Balancer uses?
> Add new RPC to transfer data block with external shell script across Datanode
> -----------------------------------------------------------------------------
>
> Key: HDFS-15022
> URL: https://issues.apache.org/jira/browse/HDFS-15022
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Reporter: Yang Yun
> Assignee: Yang Yun
> Priority: Minor
> Attachments: HDFS-15022.patch, HDFS-15022.patch,
> link_block_across_datanode.pdf
>
>
> Replicating data block is expensive when some Datanodes are down, especially
> for slow storage. Add a new RPC to replicate block with external shell script
> across datanode. User can choose more effective way to copy block files.
> In our setup, Archive volume are configured to remote reliable storage. we
> just add a new link file in new datanode to the remote file when do
> replication.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
