rhalm opened a new pull request, #10901:
URL: https://github.com/apache/ozone/pull/10901
## What changes were proposed in this pull request?
Improves snapshot diff progress granularity by letting clients track which
stage a diff job is in and adding more informative logs. Main changes:
- Exposes substatus and progress percentage (when available) to clients in
snapshot diff responses, and fixes minor bugs causing incorrect values:
- Wires `subStatus` and `progressPercent` through the proto layer and
CLI output (with a minor formatting fix)
- Adds a new substatus for the FSO path resolution stage
(`PATH_RESOLUTION_FSO`)
- Fixes progress percentage carrying over between stages by resetting
progress to 0 at the start
- Fixes a bug where the substatus would remain stuck on
`SST_FILE_DELTA_*` instead of switching to `OBJECT_ID_MAP_GEN_*` during object
ID map generation (which now uses `OBJECT_ID_MAP_GEN_OBS` for OBS and FSO file
table, `OBJECT_ID_MAP_GEN_FSO` for FSO directory table)
- Improves logging for snapshot diff processing:
- Adds one summary log per stage with item count and elapsed time
- Adds a warn log when DAG-based delta computation returns no result and
falls back to full diff
**Before / After**
Example CLI output when polling an in-progress snapshot diff before this
change:
```bash
$ ozone sh snapshot diff --get-report /vol/bucket snap1 snap2
Snapshot diff job is IN_PROGRESS. Please retry after 60000 ms.
$ ozone sh snapshot diff --get-report --json /vol/bucket snap1 snap2
{
"snapshotDiffReport" : {
...
},
"jobStatus" : "IN_PROGRESS",
"waitTimeInMs" : 60000
}
```
And after:
```bash
$ ozone sh snapshot diff --get-report /vol/bucket snap1 snap2
Snapshot diff job is IN_PROGRESS. Please retry after 60000 ms.
SubStatus : OBJECT_ID_MAP_GEN_FSO
Keys Processed Estimated Percentage : 25.0
$ ozone sh snapshot diff --get-report --json /vol/bucket snap1 snap2
{
"snapshotDiffReport" : {
...
},
"jobStatus" : "IN_PROGRESS",
"waitTimeInMs" : 60000,
"subStatus" : "OBJECT_ID_MAP_GEN_FSO",
"progressPercent" : 25.0
}
```
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-14876
## How was this patch tested?
- Updated and added unit tests
- Manually verified the changes on a local deployment
- CI pass: https://github.com/rhalm/ozone/actions/runs/30462314731
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
