[
https://issues.apache.org/jira/browse/HDDS-16282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Huang Kuan Hao updated HDDS-16282:
----------------------------------
Description:
The S3 multi-object DELETE handler (up to 1000 keys/request) removes each
successfully-deleted key from the deleteKeys ArrayList inside the per-object
loop — O remove x n = O(n^2):
for (DeleteObject d : request.getObjects()) {
...
if (deleted)
{ deleteKeys.remove(d.getKey()); // O(n) on ArrayList ... }
}
...
message.getParams().put("failedDeletes", deleteKeys.toString());
The removals are how the failedDeletes audit set is computed, so they can't
just be dropped. Fix: collect failed keys in a separate list and build the
audit string from it:
List<String> failedKeys = new ArrayList<>();
for (DeleteObject d : request.getObjects())
{ ... if (deleted) \\{ if (!request.isQuiet()) result.addDeleted(new
DeletedObject(d.getKey())); }
else \{ failedKeys.add(d.getKey()); result.addError(...); }
}
...
message.getParams().put("failedDeletes", failedKeys.toString());
Behavior unchanged (same failedDeletes content, same response).
File: hadoop-ozone/s3gateway/.../endpoint/BucketEndpoint.java:361
was:
The S3 multi-object DELETE handler (up to 1000 keys/request) removes each
successfully-deleted key from the deleteKeys ArrayList inside the per-object
loop — O(n) remove x n = O(n^2):
for (DeleteObject d : request.getObjects()) {
...
if (deleted) {
deleteKeys.remove(d.getKey()); // O(n) on ArrayList
...
}
}
...
message.getParams().put("failedDeletes", deleteKeys.toString());
The removals are how the failedDeletes audit set is computed, so they can't
just be dropped. Fix: collect failed keys in a separate list and build the
audit string from it:
List<String> failedKeys = new ArrayList<>();
for (DeleteObject d : request.getObjects()) {
...
if (deleted) \{ if (!request.isQuiet()) result.addDeleted(new
DeletedObject(d.getKey())); }
else \{ failedKeys.add(d.getKey()); result.addError(...); }
}
...
message.getParams().put("failedDeletes", failedKeys.toString());
Behavior unchanged (same failedDeletes content, same response).
File: hadoop-ozone/s3gateway/.../endpoint/BucketEndpoint.java:361
> Avoid O(n^2) key removal in S3 DeleteObjects
> --------------------------------------------
>
> Key: HDDS-16282
> URL: https://issues.apache.org/jira/browse/HDDS-16282
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Huang Kuan Hao
> Priority: Major
>
> The S3 multi-object DELETE handler (up to 1000 keys/request) removes each
> successfully-deleted key from the deleteKeys ArrayList inside the per-object
> loop — O remove x n = O(n^2):
> for (DeleteObject d : request.getObjects()) {
> ...
> if (deleted)
> { deleteKeys.remove(d.getKey()); // O(n) on ArrayList ... }
> }
> ...
> message.getParams().put("failedDeletes", deleteKeys.toString());
> The removals are how the failedDeletes audit set is computed, so they can't
> just be dropped. Fix: collect failed keys in a separate list and build the
> audit string from it:
> List<String> failedKeys = new ArrayList<>();
> for (DeleteObject d : request.getObjects())
> { ... if (deleted) \\{ if (!request.isQuiet()) result.addDeleted(new
> DeletedObject(d.getKey())); }
> else \{ failedKeys.add(d.getKey()); result.addError(...); }
> }
> ...
> message.getParams().put("failedDeletes", failedKeys.toString());
> Behavior unchanged (same failedDeletes content, same response).
> File: hadoop-ozone/s3gateway/.../endpoint/BucketEndpoint.java:361
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
