[
https://issues.apache.org/jira/browse/HDFS-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15514336#comment-15514336
]
Jason Kace commented on HDFS-10467:
-----------------------------------
[~subru], than you for the feedback!
1) Using jcache for the query caches is a good idea. A TODO I have is to
increase the scalability of the caches and/or to prune older entries. jcache
seems to handle these well. I'll check out YARN to see if there is a cache
manager we can reuse. For the internal caches of state store records, I'm not
convinced jcache provides any benefits as these caches are closely synchronized
with internal data structures such as the tree representation of the mount
table, etc.
2) I'll work on curator for ZK. It will simplify the codebase and connection
management.
3) I'll add versioning. For HDFS federation, there are multiple APIs
implemented in different classes, I recommend that each of these are versioned
(i.e. Registration, MountTable, RouterState, Rebalancer, etc). The driver
interface is separate from the interface APIs and should also be versioned.
Each of the data records and/or API request/response objects can potentially be
versioned, but I think it is best to keep their version tied to the interface
API as each has a 1:many relationship between the interface:object.
> Router-based HDFS federation
> ----------------------------
>
> Key: HDFS-10467
> URL: https://issues.apache.org/jira/browse/HDFS-10467
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: fs
> Affects Versions: 2.7.2
> Reporter: Inigo Goiri
> Assignee: Inigo Goiri
> Attachments: HDFS Router Federation.pdf, HDFS-10467.PoC.001.patch,
> HDFS-10467.PoC.patch, HDFS-Router-Federation-Prototype.patch
>
>
> Add a Router to provide a federated view of multiple HDFS clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
