[
https://issues.apache.org/jira/browse/HDDS-10629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devesh Kumar Singh updated HDDS-10629:
--------------------------------------
Summary: Recon - Enriched API for listing of OBS, Legacy bucket keys with
recursive support for FSO bucket (was: Recon - API for listing of OBS, Legacy
bucket keys with recursive support for FSO bucket)
> Recon - Enriched API for listing of OBS, Legacy bucket keys with recursive
> support for FSO bucket
> -------------------------------------------------------------------------------------------------
>
> Key: HDDS-10629
> URL: https://issues.apache.org/jira/browse/HDDS-10629
> Project: Apache Ozone
> Issue Type: Task
> Components: Ozone Recon
> Affects Versions: 1.5.0
> Reporter: Devesh Kumar Singh
> Assignee: Devesh Kumar Singh
> Priority: Major
>
> * API to retrieve list of keys in the ozone. Probably allow filtering of keys
> by bucket, replication_type, age, size, limit and key prefix.
> *
> h5. This API will leverage the existing design and implementation of
> NamespaceSummary data built by Recon. Existing NSSummary data is missing a
> few data fields and will need to be extended while implementing the API.
> #
> h5. By default, API will provide a list of keys based on the path provided.
> E.g. /volume/bucket/
> #
> h5. API will also accept parameters to filter the list of keys by path
> (volume/bucket/dir/sub-dir), replication type, age, key size, start, count
> and start prefix of key.
> #
> h5. Default values of API parameters if not provided:
> #
> h5. path - /
> #
> h5. recursive - true (Applicable only for FSO buckets)
> #
> h5. replication_type - RATIS
> #
> h5. age - null and filter will not be applied, so list out keys irrespective
> of age.
> #
> h5. key_size - 0 bytes, which means all keys greater than zero bytes will be
> listed, effectively all.
> #
> h5. start - 1 (inclusive)
> #
> h5. count - 1000
> #
> h5. Behavior of API:
> #
> h5. For OBS bucket - list out *_count_* number of keys starting from
> *_start_* value, existing in a bucket based on further filter criteria.
> #
> h5. For FSO bucket - list out *_count_* number of keys starting from
> *_start_* value number of keys recursively, i.e. on immediate path as well as
> all files in all directories and sub-directories on the provided path.
> #
> h5. For LEGACY bucket -
> # FS path enabled - same like FSO bucket behavior.
> # FS Path not enabled - same like OBS bucket behavior.
> This API will implement pagination support using *start* and *count params.*
>
> h3. Get List of All Keys:
> **
> * GET /api/v1/keys/list
> * API params:
> *
> h5. path
> *
> h5. replication_type
> *
> h5. age in epoch ms
> *
> h5. start
> *
> h5. count
> *
> h5. key_size
> * Input Request:
> **
> api/v1/namespace/listKeys?path=/volume1/fso-bucket/dir1/&start=5&count=105
> **
> * Output Response
> **
> {
> "status": "OK",
> "path": "/volume1/fso-bucket/dir1",
> "size": 62914560,
> "sizeWithReplica": -1,
> "subPathCount": 3,
> "subPaths": [
> {
> "path": "/volume1/fso-bucket/dir1/dir2",
> "size": 41943040,
> "sizeWithReplica": -1,
> "isKey": false,
> "subPathCount": 3,
> "subPaths": [
> {
> "key": true,
> "path": "/volume1/fso-bucket/dir1/dir2/file1",
> "size": 41943040,
> "sizeWithReplica": -1,
> "isKey": true,
> "subPathCount": 0,
> **
> },
> {
> "key": true,
> "path": "/volume1/fso-bucket/dir1/dir2/file2",
> "size": 41943040,
> "sizeWithReplica": -1,
> "isKey": true,
> "subPathCount": 0,
> **
> },
> {
> "key": true,
> "path": "/volume1/fso-bucket/dir1/dir2/file3",
> "size": 41943040,
> "sizeWithReplica": -1,
> "isKey": true,
> "subPathCount": 0,
> **
> }
> ]
> "replicaType": "RATIS"
> },
> {
> "key": true,
> "path": "/volume1/fso-bucket/dir1/file1",
> "size": 10485760,
> "sizeWithReplica": -1,
> "isKey": true,
> "replicaType": "EC"
> },
> {
> "key": true,
> "path": "/volume1/fso-bucket/dir1/testfile",
> "size": 10485760,
> "sizeWithReplica": -1,
> "isKey": true,
> "replicaType": "RATIS"
> }
> ],
> "sizeDirectKey": 20971520
> }
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
