rangareddy commented on issue #17311:
URL: https://github.com/apache/hudi/issues/17311#issuecomment-5507012017
Closing this as resolved, under a different ticket rather than this one.
The behaviour reported here, an UPDATE silently leaving the partition column
at its old value, cannot occur on `master`. HUDI-8823 ("Ban update query from
changing partition column value", **Closed/Fixed**) settled it by rejecting the
statement outright, and that landed in `6b4c8f3308dd` ("[HUDI-8823] Ban update
from updating primary key and partition key", #12587, 2025-01-14), six weeks
after this ticket was filed.
On current `master` the statement in the description throws
`HoodieAnalysisException` during analysis, before any write:
https://github.com/apache/hudi/blob/a2788eef922e2375432ead3fe50af7fe5f55faf7/hudi-spark-datasource/hudi-spark3-common/src/main/scala/org/apache/spark/sql/hudi/command/UpdateHoodieTableCommand.scala#L132-L139
The Spark 4 copy carries the same check, and `TestUpdateTable.scala:419-428`
pins the behaviour, so the silent-wrong-result path is gone rather than merely
unobserved.
To be precise about what was and was not resolved: the data-correctness bug
is fixed, the capability is not. Moving a record between partitions from an
UPDATE statement is still unsupported by design. Anyone who wants that should
open a **new feature request** referencing HUDI-8823, so a reviewer can see the
ban is being lifted deliberately rather than treat it as a regression fix.
Reopening this issue for that purpose would obscure the distinction.
Two notes for anyone arriving here:
- Cross-partition record movement from SQL is supported today via MERGE INTO
with a global index and the matching `update.partition.path` config enabled.
See `MergeIntoHoodieTableCommand.scala:306-317` and the index-to-config map at
`:1197-1201`.
- The rejection message ends with "Please remove the assignment clause to
avoid the error", which does not name that alternative. Worth a small follow-up
so the next person does not need this thread.
HUDI-8568 in JIRA is still Open and assigned; it needs resolving there as
well, pointing at HUDI-8823.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
