sadanand48 commented on code in PR #7365:
URL: https://github.com/apache/ozone/pull/7365#discussion_r1889807433
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/DirectoryDeletingService.java:
##########
@@ -217,13 +221,13 @@ public BackgroundTaskResult call() {
LOG.debug("Running DirectoryDeletingService");
}
isRunningOnAOS.set(true);
- long rnCnt = getRunCount().incrementAndGet();
long dirNum = 0L;
long subDirNum = 0L;
long subFileNum = 0L;
long remainNum = pathLimitPerTask;
- int consumedSize = 0;
- List<PurgePathRequest> purgePathRequestList = new ArrayList<>();
+ long remainingBufLimit = ratisByteLimit;
+ List<List<PurgePathRequest>> purgeRequestListBatches = new
ArrayList<>();
Review Comment:
@sumitagrawl Thanks for the suggestion . Agree there would be concurrency
issue if multiple batches are sent to satisfy the taskLimit, Initially I had
written code to solve this by maintaining the last processed key in each batch
which can be used as a startKey in the next batch but the code got too complex
and debugging the test failures gave me a hard time. I agree too that we could
only use the ratisByteLimit as the decisive parameter to limit the number of
dirs/keys. @errose28 The description in the jira suggests splitting into
batches to satisfy the task limit. Are we okay if we just follow the
ratisByteLimit and not follow the taskLimit here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
