snleee commented on a change in pull request #8067:
URL: https://github.com/apache/pinot/pull/8067#discussion_r796117810
##########
File path:
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/TableConfigUtils.java
##########
@@ -857,4 +860,53 @@ public static void verifyHybridTableConfigs(String
rawTableName, TableConfig off
public enum ValidationType {
ALL, TASK, UPSERT
}
+
+ /**
+ * Helper method to check is EmptySegmentPruner for a TableConfig.
+ * @param tableConfig Input table config.
+ */
+ public static boolean needsEmptySegmentPruner(TableConfig tableConfig) {
+ if (isKinesisConfigured(tableConfig)) {
+ return true;
+ }
+ RoutingConfig routingConfig = tableConfig.getRoutingConfig();
+ if (routingConfig == null) {
+ return false;
+ }
+ List<String> segmentPrunerTypes = routingConfig.getSegmentPrunerTypes();
+ if (segmentPrunerTypes == null || segmentPrunerTypes.isEmpty()) {
+ return false;
+ }
+ for (String segmentPrunerType : segmentPrunerTypes) {
+ if
(RoutingConfig.EMPTY_SEGMENT_PRUNER_TYPE.equalsIgnoreCase(segmentPrunerType)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ private static boolean isKinesisConfigured(TableConfig tableConfig) {
+ IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+ if (indexingConfig != null) {
+ Map<String, String> streamConfig = indexingConfig.getStreamConfigs();
+ if (streamConfig != null && KinesisConfig.STREAM_TYPE.equals(
Review comment:
After discussion with @Jackie-Jiang and @npawar, it's probably better to
put the following hard-coded value in `TableConfigUtils` than pulling the
entire `pinot-kinesis` module. Let's add the comments on why we use hard-coded
value.
```
// Please add the explanation on why we use this instead of
KinesisConfig.STREAM_TYPE
private static final String KINESIS_STREAM_TYPE = "kinesis";
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
