Eric Marnadi created SPARK-61310:
------------------------------------
Summary: Decide streaming source evolution once per query run
Key: SPARK-61310
URL: https://issues.apache.org/jira/browse/SPARK-61310
Project: Spark
Issue Type: Improvement
Components: Structured Streaming
Affects Versions: 5.0.0
Reporter: Eric Marnadi
Decide once per query run whether a streaming query tracks its sources by name:
the value recorded in the checkpoint wins, and a new query takes
spark.sql.streaming.queryEvolution.enableSourceEvolution. Write the decision
into the query's cloned session, so that every offset log entry records it, and
pass it to CheckpointVersionManager instead of having it read the config. No
behavior change.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
