[
https://issues.apache.org/jira/browse/HDDS-8940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17745257#comment-17745257
]
Hemant Kumar commented on HDDS-8940:
------------------------------------
I checked the logs on the recent RMT testing setup after recent major fixes we
did. And could not find any occurrence of the [warn
message|https://github.com/apache/ozone/blob/7a19afa71401e356cd86a99648e94278ea2850c8/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/SnapshotDiffManager.java#L1240]
added in HDDS-8935.
Recent changes:
1. Fallback to full diff if getDetlaFiles from compaction DAG fails
[HDDS-8935|https://issues.apache.org/jira/browse/HDDS-8935].
2. SstFilteringService and other snapshot code path should use SnapshotCache
[HDDS-8938|https://issues.apache.org/jira/browse/HDDS-8938].
3. Pause the compaction log append if tarball creation is in progress
[HDDS-9004|https://issues.apache.org/jira/browse/HDDS-9004].
4. Make RocksDB delete SST File API
synchronous[HDDS-8987|https://issues.apache.org/jira/browse/HDDS-8987].
Either the above changes have fixed it or it is not reproducible any more. We
can close this for now or leave it on low priority till we see another
occurrence of it.
> SST files are missing on optimized snapDiff path.
> -------------------------------------------------
>
> Key: HDDS-8940
> URL: https://issues.apache.org/jira/browse/HDDS-8940
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Hemant Kumar
> Assignee: Hemant Kumar
> Priority: Major
> Attachments: HDDS-8940_Compaction_Dag.png
>
>
> While running snapDiff, we are seeing SST files missing on optimized snapDiff
> path.
> {code}
> 2023-06-23 19:59:16,323 [snapshot-diff-job-thread-id-14] ERROR
> org.apache.hadoop.ozone.om.snapshot.SnapshotDiffManager: Caught checked
> exception during diff report generation for volume: volume1 bucket: bucket1,
> fromSnapshot: alma2 and toSnapshot:
> cm-tmp-0ae3d532-237d-4df2-83f9-4844d153521e
> java.io.FileNotFoundException: Can't find SST file: 010788.sst
> at
> org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.getAbsoluteSstFilePath(RocksDBCheckpointDiffer.java:654)
> at
> org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.filterRelevantSstFilesFullPath(RocksDBCheckpointDiffer.java:949)
> at
> org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.getSSTDiffList(RocksDBCheckpointDiffer.java:933)
> at
> org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.getSSTDiffListWithFullPath(RocksDBCheckpointDiffer.java:868)
> at
> org.apache.hadoop.ozone.om.snapshot.SnapshotDiffManager.getDeltaFiles(SnapshotDiffManager.java:929)
> at
> org.apache.hadoop.ozone.om.snapshot.SnapshotDiffManager.getDeltaFilesAndDiffKeysToObjectIdToKeyMap(SnapshotDiffManager.java:793)
> at
> org.apache.hadoop.ozone.om.snapshot.SnapshotDiffManager.generateSnapshotDiffReport(SnapshotDiffManager.java:721)
> at
> org.apache.hadoop.ozone.om.snapshot.SnapshotDiffManager.lambda$0(SnapshotDiffManager.java:565)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
