Gengliang Wang created SPARK-60718:
--------------------------------------
Summary: Check whether an expression accepts an indeterminate
collation before typing its children
Key: SPARK-60718
URL: https://issues.apache.org/jira/browse/SPARK-60718
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 5.0.0
Reporter: Gengliang Wang
CollationTypeCoercion checks every expression it visits for an input with
indeterminate collation. The check computes the data type of every child first,
and only then asks whether the expression accepts such an input. Almost every
expression does, so the child types are thrown away.
For the WithField and DropField operations of UpdateFields, asking for the data
type throws an internal error, which the check catches and discards. So
analyzing a plan that updates struct fields builds and discards one exception,
with its stack trace, for each such operation every time the check visits the
UpdateFields.
Check the expression first, and compute the child types only when the
expression does not accept an indeterminate collation. The verdict is unchanged
for every expression.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
