klsince commented on code in PR #10553:
URL: https://github.com/apache/pinot/pull/10553#discussion_r1169266722
##########
pinot-common/src/main/java/org/apache/pinot/common/utils/config/TableConfigUtils.java:
##########
@@ -331,6 +339,96 @@ public static void
convertFromLegacyTableConfig(TableConfig tableConfig) {
validationConfig.setSegmentPushType(null);
}
+ /**
+ * Helper method to create a new TableConfig by overwriting the original
TableConfig with tier specific configs, so
+ * that the consumers of TableConfig don't have to handle tier overwrites
themselves. To begin with, we only
+ * consider to overwrite the index configs in `tableIndexConfig` and
`fieldConfigList`, e.g.
+ *
+ * {
+ * "tableIndexConfig": {
+ * ... // configs allowed in IndexingConfig, for default tier
+ * "tierOverwrites": {
+ * "hotTier": {...}, // configs allowed in IndexingConfig, for hot tier
+ * "coldTier": {...} // configs allowed in IndexingConfig, for cold
tier
+ * }
+ * }
+ * "fieldConfigList": [
+ * {
+ * ... // configs allowed in FieldConfig, for default tier
+ * "tierOverwrites": {
+ * "hotTier": {...}, // configs allowed in FieldConfig, for hot tier
+ * "coldTier": {...} // configs allowed in FieldConfig, for cold tier
+ * }
+ * },
+ * ...
+ * ]
+ * }
Review Comment:
The precedence between using fieldConfigList or tableIndexConfig is defined
by IndexType.getConfig() from index-spi, which basically favors
fieldConfigList. But this is the detail of IndexType.getConfig() and may get
changed as needed in future. So changes in this PR tried to decouple
IndexType.getConfig() from the logic of applying tier specific configs.
But for IndexType.getConfig() to access tier specific configs, we make a
temp TableConfig by copying tier specific configs from `tierOverwrites` out and
overwrite those defined for default tier, and then pass the TableConfig object
to IndexType.getConfig().
> ... is the instruction to just repeat whatever json fields...
mostly right, but only for `top-level` config keys defined in IndexingConfig
or FieldConfig, e.g. StarTreeIndexConfig is overwritten as a whole, instead of
overwriting the inner fields of StarTreeIndexConfig.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
