Sean Mackrory commented on HDFS-9806:

Could someone more familiar with the design here comment on any anticipated 
impact to NameNode scalability? If this feature is included but a user chooses 
not to use it (i.e. there are no storages of type PROVIDED) - is there any 
impact to memory consumed by the NameNode? I've read the design doc and some of 
the patches - I *think* things are good but wanting to be sure by someone more 
familiar with the implementation and who might have even tested that already...

> Allow HDFS block replicas to be provided by an external storage system
> ----------------------------------------------------------------------
>                 Key: HDFS-9806
>                 URL: https://issues.apache.org/jira/browse/HDFS-9806
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Chris Douglas
>         Attachments: HDFS-9806-design.001.pdf, HDFS-9806-design.002.pdf
> In addition to heterogeneous media, many applications work with heterogeneous 
> storage systems. The guarantees and semantics provided by these systems are 
> often similar, but not identical to those of 
> [HDFS|https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/filesystem/index.html].
>  Any client accessing multiple storage systems is responsible for reasoning 
> about each system independently, and must propagate/and renew credentials for 
> each store.
> Remote stores could be mounted under HDFS. Block locations could be mapped to 
> immutable file regions, opaque IDs, or other tokens that represent a 
> consistent view of the data. While correctness for arbitrary operations 
> requires careful coordination between stores, in practice we can provide 
> workable semantics with weaker guarantees.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to