robertzych commented on code in PR #10463:
URL: https://github.com/apache/pinot/pull/10463#discussion_r1218639989
##########
pinot-core/src/main/java/org/apache/pinot/core/common/MinionConstants.java:
##########
@@ -136,4 +136,18 @@ public static class SegmentGenerationAndPushTask {
public static final String CONFIG_NUMBER_CONCURRENT_TASKS_PER_INSTANCE =
"SegmentGenerationAndPushTask.numConcurrentTasksPerInstance";
}
+
+ public static class UpsertCompactionTask {
+ public static final String TASK_TYPE = "UpsertCompactionTask";
+ /**
+ * The time period to wait before picking segments for this task
+ * e.g. if set to "2d", no task will be scheduled for a time window
younger than 2 days
+ */
+ public static final String BUFFER_TIME_PERIOD_KEY = "bufferTimePeriod";
+ /**
+ * The maximum amount of old records allowed for an existing segment.
+ * e.g. if the current amount surpasses 2500000, then the segment will be
compacted
+ */
+ public static final String INVALID_RECORDS_THRESHOLD =
"invalidRecordsThreshold";
Review Comment:
@snleee @Jackie-Jiang @ankitsultana Yes, it would be easy to calculate this
percentage as we now have both the invalid record count and the total docs for
the segment. Would 10% be a good default as a maximum threshold?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
