smengcl commented on code in PR #4811:
URL: https://github.com/apache/ozone/pull/4811#discussion_r1225214505
##########
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/fs/ozone/TestSnapshotDeletingService.java:
##########
@@ -342,17 +384,55 @@ public void testSnapshotWithFSO() throws Exception {
assertTableRowCount(snapshotInfoTable, 1);
assertTableRowCount(renamedTable, 4);
assertTableRowCount(deletedDirTable, 3);
- assertTableRowCount(deletedTable, 10);
+
+ OmSnapshot snap1 = (OmSnapshot) om.getOmSnapshotManager()
+ .checkForSnapshot(VOLUME_NAME, BUCKET_NAME_TWO,
+ getSnapshotPrefix("snap1"), true);
+ Table<String, OmKeyInfo> snap1KeyTable =
+ snap1.getMetadataManager().getFileTable();
+ try (TableIterator<String, ? extends Table.KeyValue<String,
+ RepeatedOmKeyInfo>> iterator = deletedTable.iterator()) {
+ while (iterator.hasNext()) {
+ Table.KeyValue<String, RepeatedOmKeyInfo> next = iterator.next();
+ String activeDBDeletedKey = next.getKey();
+ if (activeDBDeletedKey.matches(".*/key1.*")) {
+ RepeatedOmKeyInfo activeDBDeleted = next.getValue();
+ OMMetadataManager metadataManager =
+ cluster.getOzoneManager().getMetadataManager();
+ GenericTestUtils.waitFor(() -> activeDBDeleted.getOmKeyInfoList()
+ .size() == 1, 100, 2000);
Review Comment:
I don't think this line would work as intended?
IIUC after `next.getValue()` is called above, the value is already
deserialized into the local variable `activeDBDeleted`. And the local variable
won't change even if the actual value in DB has been changed.
In the worst case it would just be wasting 2 seconds waiting for a variable
that isn't changed in this case.
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/PendingKeysDeletion.java:
##########
Review Comment:
Missing license header
https://github.com/apache/ozone/actions/runs/5249532132/jobs/9482499170#step:6:839
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/snapshot/OMSnapshotMoveDeletedKeysResponse.java:
##########
@@ -187,5 +187,35 @@ private RepeatedOmKeyInfo
createRepeatedOmKeyInfo(List<KeyInfo> keyInfoList)
return result;
}
+
+ private RepeatedOmKeyInfo createRepeatedOmKeyInfo(
+ SnapshotMoveKeyInfos snapshotMoveKeyInfos,
+ OMMetadataManager metadataManager) throws IOException {
+ String dbKey = snapshotMoveKeyInfos.getKey();
+ List<KeyInfo> keyInfoList = snapshotMoveKeyInfos.getKeyInfosList();
+ // When older version of keys are moved to the next snapshot's deletedTable
+ // The newer version might also be in the next snapshot's deletedTable and
+ // it might overwrite. This is to avoid that and also avoid having
+ // orphans blocks.
+ RepeatedOmKeyInfo result = metadataManager.getDeletedTable().get(dbKey);
+
+ for (KeyInfo keyInfo : keyInfoList) {
+ OmKeyInfo omKeyInfo = OmKeyInfo.getFromProtobuf(keyInfo);
+ if (result == null) {
+ result = new RepeatedOmKeyInfo(omKeyInfo);
+ } else if (!isSameAsLatestOmKeyInfo(omKeyInfo, result)) {
+ result.addOmKeyInfo(omKeyInfo);
+ }
+ }
+
+ return result;
+ }
+
+ private boolean isSameAsLatestOmKeyInfo(OmKeyInfo omKeyInfo,
+ RepeatedOmKeyInfo result) {
+ int size = result.getOmKeyInfoList().size();
+ OmKeyInfo keyInfoFromRepeated = result.getOmKeyInfoList().get(size - 1);
Review Comment:
Assert `size > 0` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
