Szehon Ho created SPARK-58940:
---------------------------------
Summary: Add migration notes for Declarative Pipelines case
sensitivity
Key: SPARK-58940
URL: https://issues.apache.org/jira/browse/SPARK-58940
Project: Spark
Issue Type: Sub-task
Components: Declarative Pipelines
Affects Versions: 5.0.0
Reporter: Szehon Ho
SPARK-58517 made Declarative Pipelines honor spark.sql.caseSensitive when
inferring and evolving pipeline table schemas. That is a user-facing behavior
change relative to released 4.1.x and 4.2.x, which always compared column names
case-sensitively on these paths, and it also introduces a new error condition,
CONFLICTING_PIPELINE_FLOW_CASE_SENSITIVITY, when the flows writing to one table
disagree on the effective value.
The parent PR described the change as affecting only unreleased branches, so no
migration note was added. Add the missing notes to the 'Upgrading from Spark
SQL 4.2 to 4.3' section of docs/sql-migration-guide.md.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
