[
https://issues.apache.org/jira/browse/HDFS-15714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343771#comment-17343771
]
Bhavik Patel commented on HDFS-15714:
-------------------------------------
Yes, [~PhiloHe] you are correct, many factors we have to consider.
{code:java}
Please note that remote metadata is pulled only once for a
mount, so remote data changes cannot be reflected to HDFS side. To make the
metadata in
remote side and HDFS side consistent, user can re-mount remote storage, i.e.,
remove mount,
then add that mount again.
{code}
I am thinking on this point of design: If any Data added/updated at the remote
end then to pull that data at the Hdfs end we have to unmount/remove the path
at the Hdfs end and then again we have to perform add mount operation, that is
something not feasible so, I thought of having merge/append mount
functionality.
Definitely, we can track this feature/functionality in another Jira.
> HDFS Provided Storage Read/Write Mount Support On-the-fly
> ---------------------------------------------------------
>
> Key: HDFS-15714
> URL: https://issues.apache.org/jira/browse/HDFS-15714
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode, namenode
> Affects Versions: 3.4.0
> Reporter: Feilong He
> Assignee: Feilong He
> Priority: Major
> Labels: pull-request-available
> Attachments: HDFS-15714-01.patch,
> HDFS_Provided_Storage_Design-V1.pdf, HDFS_Provided_Storage_Performance-V1.pdf
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> HDFS Provided Storage (PS) is a feature to tier HDFS over other file systems.
> In HDFS-9806, PROVIDED storage type was introduced to HDFS. Through
> configuring external storage with PROVIDED tag for DataNode, user can enable
> application to access data stored externally from HDFS side. However, there
> are two issues need to be addressed. Firstly, mounting external storage
> on-the-fly, namely dynamic mount, is lacking. It is necessary to get it
> supported to flexibly combine HDFS with an external storage at runtime.
> Secondly, PS write is not supported by current HDFS. But in real
> applications, it is common to transfer data bi-directionally for read/write
> between HDFS and external storage.
> Through this JIRA, we are presenting our work for PS write support and
> dynamic mount support for both read & write. Please note in the community
> several JIRAs have been filed for these topics. Our work is based on these
> previous community work, with new design & implementation to support called
> writeBack mount and enable admin to add any mount on-the-fly. We appreciate
> those folks in the community for their great contribution! See their pending
> JIRAs: HDFS-14805 & HDFS-12090.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
