w41ter commented on code in PR #25847:
URL: https://github.com/apache/doris/pull/25847#discussion_r1374141890
##########
fe/fe-core/src/main/java/org/apache/doris/fs/obj/S3ObjStorage.java:
##########
@@ -223,6 +228,52 @@ public Status deleteObject(String remotePath) {
}
}
+ @Override
+ public Status deleteObjects(String absolutePath) {
+ try {
+ S3URI baseUri = S3URI.create(absolutePath, forceHostedStyle);
+ String continuationToken = "";
+ boolean isTruncated = false;
+ long totalObjects = 0;
+ do {
+ RemoteObjects objects = listObjects(absolutePath,
continuationToken);
+ List<RemoteObject> objectList = objects.getObjectList();
+ if (!objectList.isEmpty()) {
+ Delete delete = Delete.builder()
+ .objects(objectList.stream()
+ .map(RemoteObject::getKey)
+ .map(k ->
ObjectIdentifier.builder().key(k).build())
+ .collect(Collectors.toList()))
+ .build();
+ DeleteObjectsRequest req = DeleteObjectsRequest.builder()
+ .bucket(baseUri.getBucket())
+ .delete(delete)
+ .build();
+
+ DeleteObjectsResponse resp =
getClient(baseUri.getVirtualBucket()).deleteObjects(req);
+ if (resp.errors().size() > 0) {
+ LOG.warn("{} errors returned while deleting {} objects
for dir {}",
+ resp.errors().size(), objectList.size(),
absolutePath);
+ }
+ LOG.info("{} of {} objects deleted for dir {}",
+ resp.deleted().size(), objectList.size(),
absolutePath);
+ totalObjects += objectList.size();
+ }
+
+ isTruncated = objects.isTruncated();
+ continuationToken = objects.getContinuationToken();
+ } while (isTruncated);
+ LOG.info("total delete {} objects for dir {}", totalObjects,
absolutePath);
+ return Status.OK;
+ } catch (DdlException e) {
+ return new Status(Status.ErrCode.COMMON_ERROR, "list objects for
delete objects failed: " + e.getMessage());
+ } catch (Exception e) {
Review Comment:
DdlException only throws by listObjects, catching it would provide a
detailed error message.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
