LuciferYang commented on code in PR #60075:
URL: https://github.com/apache/spark/pull/60075#discussion_r4465530493
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/GeneratedColumnExpression.scala:
##########
@@ -120,6 +121,14 @@ case class GeneratedColumnExpression(
s"is incompatible with column data type
${targetDataType.simpleString}")
}
+ // Store assignment lines up the struct fields of an auto-filled value
with the column type by
+ // name, while the check of a provided value lines them up by position, so
the two must agree.
+ if (!DataType.equalsStructurallyByName(child.dataType, targetDataType,
resolver)) {
Review Comment:
This check only runs for CREATE/REPLACE TABLE, and an ALTER on a nested
field of a generated column skips it. After creating `s STRUCT<x: INT, y: INT>
GENERATED ALWAYS AS (named_struct('x', a, 'y', b))`, a single `ALTER TABLE t
ALTER COLUMN s.y FIRST` puts the table in the state this PR rejects. I ran it
at b9c9a3936b3: rewriting an auto-filled row fails with
`CHECK_CONSTRAINT_VIOLATION`, while with a=7, b=8 the provided value
`named_struct('y', 7, 'x', 8)` (so x=8) passes.
The ALTER path exists on master too, but the description's "leaves only
expressions on which both line up the fields the same way" no longer holds
after such an ALTER, or for a table copied with `CREATE TABLE ... LIKE`. One
option is for `checkAlterTableCommand` to reject ALTERs that change the names,
order or number of fields inside a generated column. Narrowing the description
and handling ALTER in a separate JIRA works too. I'd like to hear what you
think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
