Devesh Kumar Singh created HDDS-10629:
-----------------------------------------
Summary: Recon - API for recursive listing of FSO bucket keys
Key: HDDS-10629
URL: https://issues.apache.org/jira/browse/HDDS-10629
Project: Apache Ozone
Issue Type: Task
Components: Ozone Recon
Affects Versions: 1.5.0
Reporter: Devesh Kumar Singh
Assignee: Devesh Kumar Singh
* API to retrieve list of keys in the ozone. Probably allow filtering of keys
by bucket, replication_type, age, size, limit and key prefix.
*
h5. This API will leverage the existing design and implementation of
NamespaceSummary data built by Recon. Existing NSSummary data is missing a few
data fields and will need to be extended while implementing the API.
#
h5. By default, API will provide a list of keys based on the path provided.
E.g. /volume/bucket/
#
h5. API will also accept parameters to filter the list of keys by path
(volume/bucket/dir/sub-dir), replication type, age, key size, start, count and
start prefix of key.
#
h5. Default values of API parameters if not provided:
#
h5. path - /
#
h5. recursive - true (Applicable only for FSO buckets)
#
h5. replication_type - RATIS
#
h5. age - null and filter will not be applied, so list out keys irrespective of
age.
#
h5. key_size - 0 bytes, which means all keys greater than zero bytes will be
listed, effectively all.
#
h5. start - 1 (inclusive)
#
h5. count - 1000
#
h5. Behavior of API:
#
h5. For OBS bucket - list out *_count_* number of keys starting from *_start_*
value, existing in a bucket based on further filter criteria.
#
h5. For FSO bucket - list out *_count_* number of keys starting from *_start_*
value number of keys recursively, i.e. on immediate path as well as all files
in all directories and sub-directories on the provided path.
#
h5. For LEGACY bucket -
# FS path enabled - same like FSO bucket behavior.
# FS Path not enabled - same like OBS bucket behavior.
This API will implement pagination support using *start* and *count params.*
h3. Get List of All Keys:
**
* GET /api/v1/keys/list
* API params:
*
h5. path
*
h5. replication_type
*
h5. age in epoch ms
*
h5. start
*
h5. count
*
h5. key_size
* Input Request:
**
api/v1/namespace/listKeys?path=/volume1/fso-bucket/dir1/&start=5&count=105
**
* Output Response
**
{
"status": "OK",
"path": "/volume1/fso-bucket/dir1",
"size": 62914560,
"sizeWithReplica": -1,
"subPathCount": 3,
"subPaths": [
{
"path": "/volume1/fso-bucket/dir1/dir2",
"size": 41943040,
"sizeWithReplica": -1,
"isKey": false,
"subPathCount": 3,
"subPaths": [
{
"key": true,
"path": "/volume1/fso-bucket/dir1/dir2/file1",
"size": 41943040,
"sizeWithReplica": -1,
"isKey": true,
"subPathCount": 0,
**
},
{
"key": true,
"path": "/volume1/fso-bucket/dir1/dir2/file2",
"size": 41943040,
"sizeWithReplica": -1,
"isKey": true,
"subPathCount": 0,
**
},
{
"key": true,
"path": "/volume1/fso-bucket/dir1/dir2/file3",
"size": 41943040,
"sizeWithReplica": -1,
"isKey": true,
"subPathCount": 0,
**
}
]
"replicaType": "RATIS"
},
{
"key": true,
"path": "/volume1/fso-bucket/dir1/file1",
"size": 10485760,
"sizeWithReplica": -1,
"isKey": true,
"replicaType": "EC"
},
{
"key": true,
"path": "/volume1/fso-bucket/dir1/testfile",
"size": 10485760,
"sizeWithReplica": -1,
"isKey": true,
"replicaType": "RATIS"
}
],
"sizeDirectKey": 20971520
}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
