[
https://issues.apache.org/jira/browse/SPARK-61311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-61311:
-----------------------------------
Labels: pull-request-available (was: )
> Enable streaming source evolution for queries that name a source
> ----------------------------------------------------------------
>
> Key: SPARK-61311
> URL: https://issues.apache.org/jira/browse/SPARK-61311
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Affects Versions: 5.0.0
> Reporter: Eric Marnadi
> Priority: Major
> Labels: pull-request-available
>
> DataStreamReader.name() and SQL IDENTIFIED BY are public, but using them
> fails with STREAMING_QUERY_EVOLUTION_ERROR.SOURCE_NAMING_NOT_SUPPORTED unless
> the internal config spark.sql.streaming.queryEvolution.enableSourceEvolution
> is set.
> Naming a source should enable source evolution for the query it belongs to. A
> new query with a named source starts on offset log v2 and records the mode in
> its offset metadata, a restart keeps the recorded mode, and other queries in
> the session are unaffected. Every source of such a query must be named. The
> config stays and forces source evolution for every streaming query in the
> session.
> Error messages that tell users to set the config to name sources are reworded.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
