cshuo commented on code in PR #12132:
URL: https://github.com/apache/hudi/pull/12132#discussion_r1808648628
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java:
##########
@@ -378,6 +378,14 @@ private FlinkOptions() {
.withDescription("Enables data-skipping allowing queries to leverage
indexes to reduce the search space by "
+ "skipping over files");
+ @AdvancedConfig
+ public static final ConfigOption<Boolean>
READ_PARTITION_DATA_SKIPPING_ENABLED = ConfigOptions
+ .key("read.partition.data.skipping.enabled")
Review Comment:
The main purpose here is to provide the ability to enable different indexes
separately, just in case that there are unexpected behaviors in some indexes.
Maybe we can change the code as all indexes are enabled by default when data
skipping is enabled, and introduce a new config option
'read.data.skipping.excluded.indexes' to disable some indexes if necessary.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
