snleee commented on a change in pull request #7995:
URL: https://github.com/apache/pinot/pull/7995#discussion_r783452376
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/PinotHelixResourceManager.java
##########
@@ -2811,15 +2814,36 @@ public String startReplaceSegments(String
tableNameWithType, List<String> segmen
// By here, the lineage entry is either 'IN_PROGRESS' or 'COMPLETED'.
- // When 'forceCleanup' is enabled, we need to proactively revert the
lineage entry when we find the lineage
- // entry with the same 'segmentFrom' values.
- if (forceCleanup && lineageEntry.getState() ==
LineageEntryState.IN_PROGRESS && CollectionUtils
+ // When 'forceCleanup' is enabled, we need to proactively clean up
at the following cases:
+ // 1. Revert the lineage entry when we find the lineage entry with
the same 'segmentFrom' values. This is
+ // used to un-block the segment replacement protocol if the
previous attempt failed in the middle.
+ // 2. Proactively delete the oldest data snapshot to make sure that
we only keep at most 2 data snapshots
+ // at any time in case of REFRESH use case.
+ if (forceCleanup) {
+ if (lineageEntry.getState() == LineageEntryState.IN_PROGRESS &&
CollectionUtils
.isEqualCollection(segmentsFrom,
lineageEntry.getSegmentsFrom())) {
- // Update segment lineage entry to 'REVERTED'
- updateSegmentLineageEntryToReverted(tableNameWithType,
segmentLineage, entryId, lineageEntry);
-
- // Add segments for proactive clean-up.
- segmentsToCleanUp.addAll(lineageEntry.getSegmentsTo());
+ // Update segment lineage entry to 'REVERTED'
+ updateSegmentLineageEntryToReverted(tableNameWithType,
segmentLineage, entryId, lineageEntry);
Review comment:
added logs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
