szehon-ho commented on code in PR #58632:
URL: https://github.com/apache/spark/pull/58632#discussion_r4032685838
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TableOutputResolver.scala:
##########
@@ -354,12 +369,17 @@ object TableOutputResolver extends SQLConfHelper with
Logging {
byName: Boolean,
conf: SQLConf,
addError: String => Unit,
- colPath: Seq[String]): Boolean = {
+ colPath: Seq[String],
+ deferAnsiCastValidationToRuntime: Boolean): Boolean = {
conf.storeAssignmentPolicy match {
case StoreAssignmentPolicy.STRICT | StoreAssignmentPolicy.ANSI =>
+ // Always delegate to DataTypeUtils.canWrite so structural checks
(struct field names and
+ // nullability, array/map element nullability, field counts) still
run. Only the atomic
+ // ANSI store-assignment cast check is relaxed, via
deferAnsiCastValidationToRuntime.
DataTypeUtils.canWrite(
tableName, valueType, expectedType, byName, conf.resolver,
colPath.quoted,
- conf.storeAssignmentPolicy, addError)
+ conf.storeAssignmentPolicy, addError,
+ deferAnsiCastValidationToRuntime = deferAnsiCastValidationToRuntime)
case _ =>
true
Review Comment:
**[P1] Preserve UDT field-name validation under LEGACY**
For a table that opts into LEGACY, this branch skips
`DataTypeUtils.canWrite`. A by-name append from a UDT backed by `STRUCT<b: INT,
a: INT>` into `STRUCT<a: INT, b: INT>` then reaches `unwrapUDT` and a
positional cast, silently writing `b` into `a` and vice versa. The current fix
for the earlier ANSI issue does not cover this branch. Please preserve
field-name validation under LEGACY too and add a regression test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
