[
https://issues.apache.org/jira/browse/HDFS-14805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919918#comment-16919918
]
Virajith Jalaparti commented on HDFS-14805:
-------------------------------------------
The design document for this enhancement is attached. We have developed these
enhancements on branch-3.1. We recently ported these changes to trunk ([Github
link|https://github.com/Microsoft-CISL/hadoop/commits/dynamic-mounts]). The
code here is intended to provide an idea of the changes made to implement these
enhancements. Note that this branch also has the changes for caching remote
data in HDFS (part of HDFS-13069).
> Mounting external stores in HDFS on-the-fly
> -------------------------------------------
>
> Key: HDFS-14805
> URL: https://issues.apache.org/jira/browse/HDFS-14805
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Virajith Jalaparti
> Priority: Major
> Attachments: dynamic-mounts-in-hdfs.pdf
>
>
> Provided storage (HDFS-9806) allows HDFS to address data in external storage
> systems, including cloud stores. Data mounted in this manner, seamlessly,
> appears to be part of HDFS for applications/clients. The external data can
> also be cached by HDFS on local disks and SSDs, accelerating remote data
> reads (HDFS-13069).
> However, Provided storage was originally targeted at ephemeral HDFS
> deployments in the cloud (e.g., Azure HDInsight). Long running HDFS clusters
> are common in many other scenarios which can benefit from accessing data in
> remote stores. This JIRA targets such scenarios and aims to provide the
> ability to:
> (a) Dynamically mount external stores in a HDFS cluster while supporting high
> availability.
> (b) Mount multiple remote stores simultaneously.
> (c) Reduce deployment overheads and simplify usability of Provided storage.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
