jtao15 commented on a change in pull request #7178:
URL: https://github.com/apache/incubator-pinot/pull/7178#discussion_r673296717
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/common/MinionConstants.java
##########
@@ -67,16 +68,22 @@ private MinionConstants() {
public static final String GRANULARITY_KEY = "granularity";
// Rollup aggregate function related configs
- public static final String AGGREGATE_KEY_PREFIX = "aggregate";
+ public static final String AGGREGATION_TYPE_KEY_SUFFIX =
".aggregationType";
// Merge properties related configs
- public static final String MERGE_KEY_PREFIX = "merge";
- public static final String BUFFER_TIME = "bufferTime";
+ public static final String BUFFER_TIME_PERIOD = "bufferTimePeriod";
+ public static final String BUCKET_TIME_PERIOD = "bucketTimePeriod";
+ public static final String ROUND_BUCKET_TIME_PERIOD =
"roundBucketTimePeriod";
public static final String MAX_NUM_RECORDS_PER_SEGMENT =
"maxNumRecordsPerSegment";
public static final String MAX_NUM_RECORDS_PER_TASK =
"maxNumRecordsPerTask";
// Segment name generator related configs
- public static final String MERGED_SEGMENT_NAME_KEY =
"mergedSegmentNameKey";
+ public static final String SEGMENT_NAME_PREFIX_KEY = "segmentNamePrefix";
+ public static final String SEGMENT_NAME_PREFIX = "merged";
+
+ // Window start and window end set by task generator
+ public static final String WINDOW_START_MS_KEY = "windowStartMs";
Review comment:
We will not filter the records, while the start/end time are useful to
mark the segment as merged by setting customMap as
`{mergeRollupTask.bucketGranularity : granularityLevel}`. Say we have segments:
`s1_0719_0720`, `s2_0720_0720` and we use 1 day bucket time. The first run
merge `s1_0719_0720` to `merged_s3_0719_0719` and `merged_s4_0720_0720`.
`merged_s3_0719_0719` will be marked since it's in the target window and won't
be picked for the next merge task. `merged_s4_0720_0720` will not be marked.
The second merge task will pick `merged_s4_0720_0720` and `s2_0720_0720`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
