JingsongLi commented on code in PR #3787:
URL: https://github.com/apache/paimon/pull/3787#discussion_r1705100126
##########
paimon-core/src/main/java/org/apache/paimon/utils/BranchManager.java:
##########
@@ -316,4 +388,45 @@ public List<String> branches() {
throw new RuntimeException(e);
}
}
+
+ /** Get all snapshots that are referenced by branches. */
+ public SortedMap<Snapshot, List<String>> branchesCreateSnapshots() {
+ TreeMap<Snapshot, List<String>> sortedSnapshots =
+ new TreeMap<>(Comparator.comparingLong(Snapshot::id));
+
+ for (String branchName : branches()) {
+ Snapshot branchCreateSnapshot =
+
snapshotManager.copyWithBranch(branchName).earliestSnapshot();
+ // Ignore the empty branch.
+ if (branchCreateSnapshot != null) {
+ FileStoreTable branchTable =
+ FileStoreTableFactory.create(
+ fileIO, new Path(branchPath(tablePath,
branchName)));
+ SortedMap<Snapshot, List<String>> snapshotTags =
branchTable.tagManager().tags();
+ Snapshot earliestSnapshot =
branchTable.snapshotManager().earliestSnapshot();
Review Comment:
What is difference betweend branchCreateSnapshot and this earliestSnapshot?
##########
paimon-core/src/main/java/org/apache/paimon/utils/BranchManager.java:
##########
@@ -316,4 +388,45 @@ public List<String> branches() {
throw new RuntimeException(e);
}
}
+
+ /** Get all snapshots that are referenced by branches. */
+ public SortedMap<Snapshot, List<String>> branchesCreateSnapshots() {
+ TreeMap<Snapshot, List<String>> sortedSnapshots =
+ new TreeMap<>(Comparator.comparingLong(Snapshot::id));
+
+ for (String branchName : branches()) {
+ Snapshot branchCreateSnapshot =
+
snapshotManager.copyWithBranch(branchName).earliestSnapshot();
+ // Ignore the empty branch.
+ if (branchCreateSnapshot != null) {
+ FileStoreTable branchTable =
+ FileStoreTableFactory.create(
+ fileIO, new Path(branchPath(tablePath,
branchName)));
+ SortedMap<Snapshot, List<String>> snapshotTags =
branchTable.tagManager().tags();
+ Snapshot earliestSnapshot =
branchTable.snapshotManager().earliestSnapshot();
+ if (snapshotTags.isEmpty()) {
+ // Create based on snapshotId.
+ sortedSnapshots
+ .computeIfAbsent(earliestSnapshot, s -> new
ArrayList<>())
+ .add(branchName);
+ } else {
+ Snapshot snapshot = snapshotTags.firstKey();
+ // current branch is create from tag.
+ if (earliestSnapshot.id() == snapshot.id()) {
+ List<String> tags = snapshotTags.get(snapshot);
+ checkArgument(tags.size() == 1);
+ sortedSnapshots
+ .computeIfAbsent(snapshot, s -> new
ArrayList<>())
Review Comment:
Why get snapshot from tags?
##########
paimon-core/src/main/java/org/apache/paimon/utils/SnapshotManager.java:
##########
@@ -696,6 +698,50 @@ private static int findPreviousOrEqualSnapshot(
return -1;
}
+ public static int findIndex(Snapshot targetSnapshot, List<Snapshot>
snapshotList) {
+ for (int i = 0; i < snapshotList.size(); i++) {
+ if (targetSnapshot.id() == snapshotList.get(i).id()) {
+ return i;
+ }
+ }
+ throw new RuntimeException(
+ String.format(
+ "Didn't find snapshot id '%s' in the list, this is
unexpected.",
+ targetSnapshot.id()));
+ }
+
+ public static List<Snapshot> findNearestNeighborsSnapshot(
+ Snapshot targetSnapshot, List<Snapshot> snapshotList,
SnapshotManager snapshotManager) {
+ List<Snapshot> skippedSnapshots = new ArrayList<>();
+
+ int index = SnapshotManager.findIndex(targetSnapshot, snapshotList);
+ // the nearest left neighbor snapshot.
+ if (index - 1 >= 0) {
+ skippedSnapshots.add(snapshotList.get(index - 1));
+ }
+ // the nearest right neighbor snapshot.
+ Snapshot nearestRight = snapshotManager.earliestSnapshot();
+ if (index + 1 < snapshotList.size()) {
+ Snapshot rightSnapshot = snapshotList.get(index + 1);
+ nearestRight = nearestRight.id() < rightSnapshot.id() ?
nearestRight : rightSnapshot;
+ }
+ skippedSnapshots.add(nearestRight);
+ return skippedSnapshots;
+ }
+
+ public static List<Snapshot> mergeTreeSetToList(
+ Collection<Snapshot> set1, Collection<Snapshot> set2) {
Review Comment:
Where is the tree set?
sortToList?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@paimon.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
