vvivekiyer commented on code in PR #9454:
URL: https://github.com/apache/pinot/pull/9454#discussion_r980644438
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/IndexLoadingConfig.java:
##########
@@ -215,6 +218,26 @@ private void extractFromTableConfig(TableConfig
tableConfig) {
}
}
+ /**
+ * Extracts compressionType for each column. Populates a map containing
column name as key and compression type as
+ * value. Note that only RAW forward index columns will be populated in this
map.
+ * @param tableConfig table config
+ */
+ private void extractCompressionConfigs(TableConfig tableConfig) {
Review Comment:
Existing CompresstionType is determined from the forwardIndex header.
The New (Desired) CompresionType is fetched from fieldConfig.
**Reasoning for NOT using noDictionaryConfig to determine desired
compressionType:**
I understand that we used noDictionaryConfig earlier to populate compression
settings. The reasoning for relying only on the new fieldConfig approach is
that - any user intending to use the "segmentReload with compressionChange"
feature should go down the new way of setting compressionType in fieldConfigs
instead of the old way. If we continue to add new feature support for changing
compressionType with noDictionaryConfig, we would just be adding more tech debt
that we would have to clean up later. Also, how would we resolve conflicts if
user specifies two different values in noDictionaryConfig and fieldConfig?
IMO, this doesn't go against backward compatibility because all we are
saying is we will support new features with only with fieldConfig. But, I
understand the concern and would be happy to discuss this further to reach a
conclusion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
