Jason Kace commented on HDFS-10467:

[~subru], than you for the feedback!

1) Using jcache for the query caches is a good idea.  A TODO I have is to 
increase the scalability of the caches and/or to prune older entries.  jcache 
seems to handle these well.  I'll check out YARN to see if there is a cache 
manager we can reuse.  For the internal caches of state store records, I'm not 
convinced jcache provides any benefits as these caches are closely synchronized 
with internal data structures such as the tree representation of the mount 
table, etc.

2) I'll work on curator for ZK.  It will simplify the codebase and connection 

3) I'll add versioning.  For HDFS federation, there are multiple APIs 
implemented in different classes, I recommend that each of these are versioned 
(i.e. Registration, MountTable, RouterState, Rebalancer, etc).  The driver 
interface is separate from the interface APIs and should also be versioned.  
Each of the data records and/or API request/response objects can potentially be 
versioned, but I think it is best to keep their version tied to the interface 
API as each has a 1:many relationship between the interface:object.

> Router-based HDFS federation
> ----------------------------
>                 Key: HDFS-10467
>                 URL: https://issues.apache.org/jira/browse/HDFS-10467
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: fs
>    Affects Versions: 2.7.2
>            Reporter: Inigo Goiri
>            Assignee: Inigo Goiri
>         Attachments: HDFS Router Federation.pdf, HDFS-10467.PoC.001.patch, 
> HDFS-10467.PoC.patch, HDFS-Router-Federation-Prototype.patch
> Add a Router to provide a federated view of multiple HDFS clusters.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to