[
https://issues.apache.org/jira/browse/HDFS-11569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15958233#comment-15958233
]
Weiwei Yang edited comment on HDFS-11569 at 4/6/17 9:18 AM:
------------------------------------------------------------
Hi [~anu]
Thanks for your comments, apparently I was missing the handler part, thanks for
pointing this out. Like you mentioned, I don't think this jira will implement
{{DistributedStorageHandler#listKeys}}, because the list key operation is
supposed to route to KSM first, then get container location from SCM. However
KSM is not ready yet, so lets use this jira to track the work of the container
layer.
About the pagination, you are making a good point. It looks better to simply
honor the arguments {{prefix}}, {{prevKey}} and {{maxNumOfKeys}}. The default
value for max number of keys is 1000 to avoid returning too many entries a
time. Works like following
# Client makes a listKey request to ozone front end
# Ozone handler, i.e {{DistributedStorageHandler}}, handles the request, it
gets the arguments from {{ListArgs}}
# Ozone handler looks up container locations from {{KSM}} where a range of keys
reside.
# Ozone handler reads keys from those containers via {{KeyManager}} interface.
# Ozone handler merges results from multiple containers and return that to
client.
This jira only addresses #4, rest of will need to be implemented when KSM is
ready. Please let me know if this approach looks good to you or not. Thank you.
was (Author: cheersyang):
Hi [~anu]
Thanks for your comments, apparently I missed the web handler part. I will
implement this. And I noticed the deleteKey function has been implemented in
{{KeyManagerImpl}}, but not yet implemented in {{DistributedStorageHandler}},
we need to get that done too (maybe in another jira), right?
About the pagination, you are making a good point. It looks better to simply
honor the arguments {{prefix}}, {{prevKey}} and {{maxKeys}}, send them to
container layer and return desired set of keys. That means we do not need
pagination in server side, instead we let client side to request proper size of
results. And we set {{maxKeys}} a default value 1000. Please let me know if
this approach looks good to you or not. Thank you.
> Ozone: Implement listKey function for KeyManager
> ------------------------------------------------
>
> Key: HDFS-11569
> URL: https://issues.apache.org/jira/browse/HDFS-11569
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
> Attachments: HDFS-11569-HDFS-7240.001.patch,
> HDFS-11569-HDFS-7240.002.patch, HDFS-11569-HDFS-7240.003.patch,
> HDFS-11569-HDFS-7240.004.patch, HDFS-11569-HDFS-7240.005.patch
>
>
> List keys by prefix from a container. This will need to support pagination
> for the purpose of small object support. So the listKey function returns
> something like ListKeyResult, client can iterate the object to get pagination
> results.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
