errose28 commented on code in PR #7314:
URL: https://github.com/apache/ozone/pull/7314#discussion_r1813529658
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/AbstractKeyDeletingService.java:
##########
@@ -452,9 +453,9 @@ public long optimizeDirDeletesAndSubmitRequest(long
remainNum,
LOG.info("Number of dirs deleted: {}, Number of sub-dir " +
"deleted: {}, Number of sub-files moved:" +
" {} to DeletedTable, Number of sub-dirs moved {} to " +
- "DeletedDirectoryTable, iteration elapsed: {}ms," +
+ "DeletedDirectoryTable, current iteration limit: {}, iteration
elapsed: {}ms," +
Review Comment:
Also from the Jira description, we should add a WARN log when the service
hits its limits, for example:
- Run time exceeded the interval time
- Limit per iteration was hit
- Log the config keys for limit and interval as hints for tuning.
I think this needs to be added to all the deletion services.
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/AbstractKeyDeletingService.java:
##########
@@ -452,9 +453,9 @@ public long optimizeDirDeletesAndSubmitRequest(long
remainNum,
LOG.info("Number of dirs deleted: {}, Number of sub-dir " +
"deleted: {}, Number of sub-files moved:" +
" {} to DeletedTable, Number of sub-dirs moved {} to " +
- "DeletedDirectoryTable, iteration elapsed: {}ms," +
+ "DeletedDirectoryTable, current iteration limit: {}, iteration
elapsed: {}ms," +
Review Comment:
```suggestion
"DeletedDirectoryTable, limit per iteration: {}, iteration
elapsed: {}ms," +
```
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/OpenKeyCleanupService.java:
##########
@@ -224,15 +238,22 @@ public BackgroundTaskResult call() throws Exception {
final OMResponse response = submitRequest(createCommitKeyRequest(b));
if (response != null && response.getSuccess()) {
ozoneManager.getMetrics().incNumOpenKeysHSyncCleaned();
+ if (LOG.isDebugEnabled()) {
+ StringBuilder sb = new StringBuilder();
+ for (CommitKeyRequest.Builder openKey : hsyncKeys) {
+ sb.append(openKey.getKeyArgs().getVolumeName() + "." +
openKey.getKeyArgs().getBucketName() + ": ")
Review Comment:
```suggestion
sb.append(openKey.getKeyArgs().getVolumeName() +
OZONE_URI_DELIMITER + openKey.getKeyArgs().getBucketName() + ": ")
```
Let's use `OzoneConsts.OZONE_URI_DELIMITER` instead of the dot.
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/BlockDeletingService.java:
##########
@@ -153,8 +155,8 @@ public BackgroundTaskQueue getTasks() {
metrics.incrTotalBlockChosenCount(totalBlocks);
metrics.incrTotalContainerChosenCount(containers.size());
if (containers.size() > 0) {
- LOG.debug("Queued {} blocks from {} containers for deletion",
- totalBlocks, containers.size());
+ LOG.info("Queued {} blocks from {} containers for deletion,
blocksLimit was {}, elapsed time {}ms.",
+ totalBlocks, containers.size(), blocksLimitPerInterval,
Time.monotonicNow() - startTime);
Review Comment:
Logs for the total queued is good, but we should also have a similar log for
each task per container queued here when it executes. Currently this is just
[this debug
log](https://github.com/apache/ozone/blob/624bede619004a6a2be5fe185abd40c2999ae3c1/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/statemachine/background/BlockDeletingTask.java#L413-L415).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
