[
https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235080#comment-16235080
]
Konstantin Shvachko commented on HDFS-7240:
-------------------------------------------
??I hope this addresses your concerns.??
I don't think _that_ addressed any of my concerns.
* Ozone by itself does not solve any of HDFS problems. It uses HDFS-agnostic
S3-like API, and I cannot use it on my clusters.
Unless I can convince thousands of my users to rewrite their thousands of
applications, along with the existing computational frameworks: YARN, Hive,
Pig, Spark, ...... created over the past 10 years.
* I was talking about futuristic architecture, when you start using Ozone for
block management, and rewrite NameNode to store its namespace in LevelDB. If
this is still your plan. I agree this architecture solves the objects-count
problem. But it does not solve the problem of scaling RPC requests, which is
more important to me than the # of objects, since you still cannot grow the
cluster beyond the single-NameNode's-RPC-processing capability.
> Object store in HDFS
> --------------------
>
> Key: HDFS-7240
> URL: https://issues.apache.org/jira/browse/HDFS-7240
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Jitendra Nath Pandey
> Assignee: Jitendra Nath Pandey
> Priority: Major
> Attachments: HDFS-7240.001.patch, HDFS-7240.002.patch,
> HDFS-7240.003.patch, HDFS-7240.003.patch, HDFS-7240.004.patch,
> Ozone-architecture-v1.pdf, Ozonedesignupdate.pdf, ozone_user_v0.pdf
>
>
> This jira proposes to add object store capabilities into HDFS.
> As part of the federation work (HDFS-1052) we separated block storage as a
> generic storage layer. Using the Block Pool abstraction, new kinds of
> namespaces can be built on top of the storage layer i.e. datanodes.
> In this jira I will explore building an object store using the datanode
> storage, but independent of namespace metadata.
> I will soon update with a detailed design document.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
