npawar commented on code in PR #10553:
URL: https://github.com/apache/pinot/pull/10553#discussion_r1169235226
##########
pinot-common/src/main/java/org/apache/pinot/common/utils/config/TableConfigUtils.java:
##########
@@ -331,6 +339,96 @@ public static void
convertFromLegacyTableConfig(TableConfig tableConfig) {
validationConfig.setSegmentPushType(null);
}
+ /**
+ * Helper method to create a new TableConfig by overwriting the original
TableConfig with tier specific configs, so
+ * that the consumers of TableConfig don't have to handle tier overwrites
themselves. To begin with, we only
+ * consider to overwrite the index configs in `tableIndexConfig` and
`fieldConfigList`, e.g.
+ *
+ * {
+ * "tableIndexConfig": {
+ * ... // configs allowed in IndexingConfig, for default tier
+ * "tierOverwrites": {
+ * "hotTier": {...}, // configs allowed in IndexingConfig, for hot tier
+ * "coldTier": {...} // configs allowed in IndexingConfig, for cold
tier
+ * }
+ * }
+ * "fieldConfigList": [
+ * {
+ * ... // configs allowed in FieldConfig, for default tier
+ * "tierOverwrites": {
+ * "hotTier": {...}, // configs allowed in FieldConfig, for hot tier
+ * "coldTier": {...} // configs allowed in FieldConfig, for cold tier
+ * }
+ * },
+ * ...
+ * ]
+ * }
Review Comment:
how does this work given tableIndexConfig is at table level, vs
fieldConfigList is at column level? What exactly is one overriding when adding
in tableIndexing? Or is the instruction to just repeat whatever json fields one
is interested in ?
##########
pinot-spi/src/main/java/org/apache/pinot/spi/utils/builder/TableConfigBuilder.java:
##########
@@ -460,6 +467,7 @@ public TableConfig build() {
indexingConfig.setAggregateMetrics(_aggregateMetrics);
indexingConfig.setOptimizeDictionaryForMetrics(_optimizeDictionaryForMetrics);
indexingConfig.setNoDictionarySizeRatioThreshold(_noDictionarySizeRatioThreshold);
+ indexingConfig.setTierOverwrites(_tierOverwrites);
Review Comment:
why is this only being set in indexingConfig and not in fieldConfigList? or
rather - how does one know which overwrites on is providing via the setter?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
