9aman commented on code in PR #16719:
URL: https://github.com/apache/pinot/pull/16719#discussion_r2313663715
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/retention/RetentionManager.java:
##########
@@ -483,4 +495,21 @@ private void
manageSegmentLineageCleanupForTable(TableConfig tableConfig) {
}
LOGGER.info("Segment lineage metadata clean-up is successfully processed
for table: {}", tableNameWithType);
}
+
+ private RetentionStrategy createUntrackedSegmentsRetentionStrategy(
+ SegmentsValidationAndRetentionConfig validationConfig, String
tableNameWithType) {
+ if (validationConfig.getUntrackedSegmentsRetentionTimeUnit() != null
+ && validationConfig.getUntrackedSegmentsRetentionTimeValue() != null) {
+ try {
+ return new TimeRetentionStrategy(
+
TimeUnit.valueOf(validationConfig.getUntrackedSegmentsRetentionTimeUnit().toUpperCase()),
+
Long.parseLong(validationConfig.getUntrackedSegmentsRetentionTimeValue()));
+ } catch (Exception e) {
+ LOGGER.warn("Invalid untracked segments retention time: {} {} for
table: {}, using default 3 days",
+ validationConfig.getUntrackedSegmentsRetentionTimeUnit(),
+ validationConfig.getUntrackedSegmentsRetentionTimeValue(),
tableNameWithType, e);
+ }
+ }
+ return new TimeRetentionStrategy(TimeUnit.DAYS, 3);
Review Comment:
I have added a check for table config validation.
`I’d suggest failing fast and emitting a metric/alert so that users or
support can fix the misconfiguration.`
This should not happen now. But even if it does, I think it's better not to
block the retention manager run for the table. The default values are a fair
estimate and a segment should make it's way to the ZK within 3 days if it's a
valid segment.
Keeping the behavior as it is here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
