Chris Douglas commented on HDFS-13123:

bq. hard linking across block pools as one option and even tiered storage
Yes, [~virajith] and I wrote a prototype of this with an intern for a similar 
project. Making it a proper transaction is complicated, but architecturally RBF 
is in the right place to coordinate this cleanly.

We added APIs to generate and attach an FSImage for a NN subtree. Attaching an 
image required reallocating not only the inodeIds but also the blockIds, which 
were hardlinked into a contiguous range in the destination blockId space. We 
didn't solve all the failover and edit log cases, but these seem tractable as 
long as the subtree is immutable. Without that assumption (which RBF can 
enforce/detect) thar be dragons.

> RBF: Add a balancer tool to move data across subsluter 
> -------------------------------------------------------
>                 Key: HDFS-13123
>                 URL: https://issues.apache.org/jira/browse/HDFS-13123
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Wei Yan
>            Assignee: Wei Yan
>            Priority: Major
>         Attachments: HDFS Router-Based Federation Rebalancer.pdf
> Follow the discussion in HDFS-12615. This Jira is to track effort for 
> building a rebalancer tool, used by router-based federation to move data 
> among subclusters.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to