[
https://issues.apache.org/jira/browse/HDFS-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359032#comment-16359032
]
Chris Douglas commented on HDFS-13123:
--------------------------------------
bq. hard linking across block pools as one option and even tiered storage
Yes, [~virajith] and I wrote a prototype of this with an intern for a similar
project. Making it a proper transaction is complicated, but architecturally RBF
is in the right place to coordinate this cleanly.
We added APIs to generate and attach an FSImage for a NN subtree. Attaching an
image required reallocating not only the inodeIds but also the blockIds, which
were hardlinked into a contiguous range in the destination blockId space. We
didn't solve all the failover and edit log cases, but these seem tractable as
long as the subtree is immutable. Without that assumption (which RBF can
enforce/detect) thar be dragons.
> RBF: Add a balancer tool to move data across subsluter
> -------------------------------------------------------
>
> Key: HDFS-13123
> URL: https://issues.apache.org/jira/browse/HDFS-13123
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Wei Yan
> Assignee: Wei Yan
> Priority: Major
> Attachments: HDFS Router-Based Federation Rebalancer.pdf
>
>
> Follow the discussion in HDFS-12615. This Jira is to track effort for
> building a rebalancer tool, used by router-based federation to move data
> among subclusters.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
