[
https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16239313#comment-16239313
]
Konstantin Shvachko commented on HDFS-7240:
-------------------------------------------
[~sanjay.radia], thank you for sharing the doc, your vision for Ozone
evolution, motivation, and compelling use cases.
I am glad I had generally correct understanding that you envisioned Ozone as a
block management layer for HDFS and a NameNode with partial namespace in memory.
[As I mentioned
above|https://issues.apache.org/jira/browse/HDFS-7240?focusedCommentId=16235080&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16235080]
the partial namespace architecture does not fully address the problem of
scaling RPCs on Hadoop clusters, which is the main pain point for me and I
believe everybody else running big analytics clusters.
You give three main reasons for Ozone inclusion into Hadoop.I think Ozone can
do all three as a separate project as well.
People run different systems on the same cluster along with Hadoop, e.g. HBase,
Spark. So Ozone will be yet one more.
Separate project Ozone does not prevent from using it as a scalable
block-container layer in HDFS. HDFS can always include Ozone as a dependency.
Especially if Ozone is already optimized for large IO scans.
> Object store in HDFS
> --------------------
>
> Key: HDFS-7240
> URL: https://issues.apache.org/jira/browse/HDFS-7240
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Jitendra Nath Pandey
> Assignee: Jitendra Nath Pandey
> Attachments: HDFS Scalability and Ozone.pdf, HDFS-7240.001.patch,
> HDFS-7240.002.patch, HDFS-7240.003.patch, HDFS-7240.003.patch,
> HDFS-7240.004.patch, Ozone-architecture-v1.pdf, Ozonedesignupdate.pdf,
> ozone_user_v0.pdf
>
>
> This jira proposes to add object store capabilities into HDFS.
> As part of the federation work (HDFS-1052) we separated block storage as a
> generic storage layer. Using the Block Pool abstraction, new kinds of
> namespaces can be built on top of the storage layer i.e. datanodes.
> In this jira I will explore building an object store using the datanode
> storage, but independent of namespace metadata.
> I will soon update with a detailed design document.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
