[ https://issues.apache.org/jira/browse/SPARK-36680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated SPARK-36680: ----------------------------------- Labels: pull-request-available (was: ) > Supports Dynamic Table Options for Spark SQL > -------------------------------------------- > > Key: SPARK-36680 > URL: https://issues.apache.org/jira/browse/SPARK-36680 > Project: Spark > Issue Type: Wish > Components: SQL > Affects Versions: 3.1.2 > Reporter: wang-zhun > Priority: Major > Labels: pull-request-available > > Now a DataFrame API user can implement dynamic options through the > _DataFrameReader$option_ method, but Spark SQL users cannot use. > {code:java} > DataFrameReader/AstBuilder -> UnresolvedRelation$options -> > DataSourceV2Relation$options -> SupportsRead$newScanBuilder(options) > {code} > > The table options were persisted to the Catalog and if we want to modify > that, we should use another DDL like "_ALTER TABLE ..._". But there are some > cases that user want to modify the table options dynamically just in the > query: > * JDBCTable set _fetchsize_ according to the actual situation of the table > * IcebergTable support time travel > {code:java} > spark.read > .option("snapshot-id", 10963874102873L) > .format("iceberg") > .load("path/to/table"){code} > These parameters setting is very common and ad-hoc, setting them flexibly > would promote the user experience with Spark SQL especially for Now we > support catalog expansion. > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org