[
https://issues.apache.org/jira/browse/SPARK-41524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Huanli Wang updated SPARK-41524:
--------------------------------
Priority: Minor (was: Major)
> Expose SQL confs and extraOptions separately in
> o.a.s.sql.execution.streaming.state.RocksDBConf
> -----------------------------------------------------------------------------------------------
>
> Key: SPARK-41524
> URL: https://issues.apache.org/jira/browse/SPARK-41524
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Affects Versions: 3.4.0
> Reporter: Huanli Wang
> Priority: Minor
>
> Currently the usage of *StateStoreConf* is via
> [*{{confs}}*|https://github.com/huanliwang-db/spark/blob/7671bc975f2d88ab929e4982abfe3e166fa72e35/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStoreConf.scala#L77-L78],
> which composes both SQL confs and extraOptions into one. The name of config
> for extraOptions shouldn't have to follow the name prefix of SQL conf,
> because it's not bound to the context of SQL conf.
> After differentiate SQL conf and extraOptions in {*}StateStoreConf{*}, we
> should be able to adopt more use case on operator level configs by using the
> {*}extraOptions{*}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
