[
https://issues.apache.org/jira/browse/HDFS-11918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16042067#comment-16042067
]
Xiaoyu Yao commented on HDFS-11918:
-----------------------------------
Thanks [~cheersyang] for proposing and posting the patches for this. I agree
with [~anu]'s comment above.
In addition, I also have performance concerns when changing the key schema for
KSM DB. LevelDB stores entries sorted lexicographically by keys. With the
serialized keys, it is not clear to me whether we will lose some locality for
large sequential read of records or not, like read some keys with certain
prefix. I would suggest we revisit this after other major KSM work is finished.
> Ozone: Encapsulate KSM metadata key into protobuf messages for better
> (de)serialization
> ---------------------------------------------------------------------------------------
>
> Key: HDFS-11918
> URL: https://issues.apache.org/jira/browse/HDFS-11918
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Priority: Critical
> Attachments: HDFS-11918-HDFS-7240.001.patch
>
>
> There are multiple type of keys stored in KSM database
> # Volume Key
> # Bucket Key
> # Object Key
> # User Key
> Currently they are represented as plain string with some conventions, such as
> # /volume
> # /volume/bucket
> # /volume/bucket/key
> # $user
> this approach makes it so difficult to parse volume/bucket/keys from KSM
> database. Propose to encapsulate these types of keys into protobuf messages,
> and take advantage of protobuf to serialize(deserialize) classes to byte
> arrays (and vice versa).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
