Vinish Reddy created HUDI-7523:
----------------------------------
Summary: Add HOODIE_SPARK_DATASOURCE_OPTIONS to be used in
HoodieIncrSource
Key: HUDI-7523
URL: https://issues.apache.org/jira/browse/HUDI-7523
Project: Apache Hudi
Issue Type: Improvement
Components: deltastreamer
Reporter: Vinish Reddy
Assignee: Vinish Reddy
Add a new config HOODIE_SPARK_DATASOURCE_OPTIONS which is used by the spark
dataframe reader for {{{}HoodieIncrSource{}}}, options like using
metadataTable, dataSkipping present inDataSourceOptions.scala can be passed for
efficient pruning of files.
[https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
