[
https://issues.apache.org/jira/browse/HDFS-11920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16100063#comment-16100063
]
Xiaoyu Yao commented on HDFS-11920:
-----------------------------------
Thanks [~vagarychen] for working on this. The patch looks good to me overall. A
few comments below
KeySpaceManagerProtocol.proto
Line 228: NIT: SubKeyInfo -> KeyLocation
Line 232: NIT: subeyDataOffset -> offset
Line 232: NIT: subeyDataSize -> length
Line 242: repeated KeyLocation locations
KSMSubKeyInfo.java
KsmSubKeyInfo -> KsmKeyLocation
OzoneClientImpl.java
Line 473: Do we need to create all the ChunkOutputstreams upon createKey? The
other option is to do
lazy creation upon read/write to different (offset) containers.
Similar logic in DistributedStorageHandler can be consolidated with a shared
utility functions.
OzoneKey.java
Line 41-48: we remove the single containerName/blockID from OzoneKey. Do we
need to add the list of
containers/blocks info like we do for the KsmKeyInfo?
ChunkGroupOutputStream.java
Line 30-31: we could add some state into the block write state so that readers
can detect write failures.
Line 73/118: the ChunkOutputStreamEntry can be created as needed here.
Line 132: can we flush only the containers that is being written based on the
state of blocks.
> Ozone : add key partition
> -------------------------
>
> Key: HDFS-11920
> URL: https://issues.apache.org/jira/browse/HDFS-11920
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Chen Liang
> Assignee: Chen Liang
> Attachments: HDFS-11920-HDFS-7240.001.patch,
> HDFS-11920-HDFS-7240.002.patch, HDFS-11920-HDFS-7240.003.patch,
> HDFS-11920-HDFS-7240.004.patch
>
>
> Currently, each key corresponds to one single SCM block, and putKey/getKey
> writes/reads to this single SCM block. This works fine for keys with
> reasonably small data size. However if the data is too huge, (e.g. not even
> fits into a single container), then we need to be able to partition the key
> data into multiple blocks, each in one container. This JIRA changes the
> key-related classes to support this.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
