[
https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234959#comment-16234959
]
Anu Engineer commented on HDFS-7240:
------------------------------------
[[email protected]] Thank you for the comments.
bq. For now, biggest issue I have is that OzoneException needs to become an IOE
I have filed HDFS-12755 for converting the OzoneException to an IOException.
bq. What's your scale limit? I see a single PUT for the upload, GET path > tmp
in open() . Is there a test for different sizes of file?
We have tested with different sizes from 1 byte files to 2 GB. There is no size
limit imposed by ozone architecture. However, we have always planned to follow
the S3 limit of 5 GB. We can certainly add tests for different size of files --
but creating these data files during unit tests take time. We have strived to
keep the unit tests of ozone under 4 mins so far. Large key sizes add
prohibitive unit test times. So our approach is to use Corona, which is a
load-generation tool for ozone. we run this 4 times daily with different key
sizes. It is trivial to setup and run.
For the comments on the OzoneFileSystem, I will let the appropriate person
respond.
> Object store in HDFS
> --------------------
>
> Key: HDFS-7240
> URL: https://issues.apache.org/jira/browse/HDFS-7240
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Jitendra Nath Pandey
> Assignee: Jitendra Nath Pandey
> Priority: Major
> Attachments: HDFS-7240.001.patch, HDFS-7240.002.patch,
> HDFS-7240.003.patch, HDFS-7240.003.patch, HDFS-7240.004.patch,
> Ozone-architecture-v1.pdf, Ozonedesignupdate.pdf, ozone_user_v0.pdf
>
>
> This jira proposes to add object store capabilities into HDFS.
> As part of the federation work (HDFS-1052) we separated block storage as a
> generic storage layer. Using the Block Pool abstraction, new kinds of
> namespaces can be built on top of the storage layer i.e. datanodes.
> In this jira I will explore building an object store using the datanode
> storage, but independent of namespace metadata.
> I will soon update with a detailed design document.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
