anew commented on code in PR #57722:
URL: https://github.com/apache/spark/pull/57722#discussion_r3722564823
##########
sql/pipelines/src/main/scala/org/apache/spark/sql/pipelines/graph/DataflowGraph.scala:
##########
@@ -170,15 +171,27 @@ case class DataflowGraph(
/**
* A map of the inferred schema of each table, computed by merging the
analyzed schemas
* of all flows writing to that table.
+ *
+ * The merge honors the session's `spark.sql.caseSensitive`: under
case-insensitive analysis two
+ * flows emitting column names that differ only in case contribute a single
column rather than
+ * both, which would otherwise produce a target schema the engine's own
resolver cannot
+ * disambiguate. Which of the two spellings survives follows the order the
flows are merged in,
+ * which this map does not define, so callers should not depend on a
particular casing.
Review Comment:
I am restoring this, but we have to be careful to not create wrong
expectations: We can state that the first flow's spelling wins, but that
"first" is relative to the order in which flows are resolved; but that is not
always the same order in which flows are defined in the source code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
