Subru Krishnan commented on HDFS-10467:

Thanks [~jakace] and [~goiri] for the refactored patch. I made a quick pass in 
the context of HADOOP-13378 and I think that we can represent the YARN 
{{FederationStateStore}} using the generic {{StateStoreDriver}} you guys have 
defined. Personally I prefer the push mechanism we have in YARN-3671 as it's 
much simpler than the pull mechanism proposed here though I do agree both 
achieve the same result.

A couple of comments based on my quick scan:
  * We should add versioning to the generic {{StateStoreDriver}}. Refer 
  * Use _jcache_ instead of writing custom key-caches based on 
_ConcurrentHashMaps_. In fact, I feel we can refactor the (jcache-based) cache 
 and use it across both efforts.
  * Lastly I would suggest using [Curator|http://curator.apache.org/] for 
*ZooKeeper* implementation as we have moved to it in YARN (YARN-4438 and follow 
up work).

> Router-based HDFS federation
> ----------------------------
>                 Key: HDFS-10467
>                 URL: https://issues.apache.org/jira/browse/HDFS-10467
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: fs
>    Affects Versions: 2.7.2
>            Reporter: Inigo Goiri
>            Assignee: Inigo Goiri
>         Attachments: HDFS Router Federation.pdf, HDFS-10467.PoC.001.patch, 
> HDFS-10467.PoC.patch, HDFS-Router-Federation-Prototype.patch
> Add a Router to provide a federated view of multiple HDFS clusters.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to