holdenk opened a new pull request, #59127: URL: https://github.com/apache/spark/pull/59127
### What changes were proposed in this pull request? Re-audits three `UdfSerialization.auditedTransitions` entries whose `localSuid` values describe master's class shapes, not branch-4.3's: - `Decimal`: `localSuid` updated to this build's computed `4401554505066564464`. Branch-4.3 has SPARK-57937's `div()` method but not SPARK-40945's truncate changes, so its SUID matches neither master's audited value nor the released one. - `Decimal$` and `StringHelper$`: entries removed. Without SPARK-40945 (`ROUND_DOWN`) and SPARK-58794 (CHAR/VARCHAR standardSemantics), both classes are unchanged from the released 4.0–4.2 jars, so stream and local SUIDs are equal and no transition is needed — keeping them also fails the suite's `streamSuid != localSuid` assert. CC @Yicong-Huang & @HyukjinKwon this tries to fix the backport ### Why are the changes needed? The SPARK-59708 backport copied master's audit table verbatim, and master's `localSuid`s reflect master-only class shapes. `UdfSerializationSuite` has failed deterministically on branch-4.3 since the backport landed (the backport's own CI ran the connect module and went red): - `audited transitions target this build` — local `Decimal` SUID is `4401554505066564464`, not the audited `6398599065815978361` - `audited transitions are backed by released producer descriptors` — `InvalidClassException: Decimal ... stream 1715621871942419369, local 4401554505066564464` branch-4.x is not affected: `Decimal.scala`, `StringType.scala` and `UdfSerialization.scala` are identical to master there. The alternative to the `Decimal` update is backporting SPARK-40945/SPARK-58794; those are features and a release-management call, not part of a build fix. ### Does this PR introduce _any_ user-facing change? No. The removed entries were dead (their audited `localSuid` could never match this build). ### How was this patch tested? - Computed the actual SUIDs of all 24 audited classes from the compiled 4.3.1-SNAPSHOT jars (`ObjectStreamClass.lookup`): exactly these three mismatched; the other 21 match. - `build/sbt -Phive 'connect/testOnly *UdfSerializationSuite'` — 16/16, including the two tests failing in branch CI. The released-descriptor test validates the audit against the real 4.0.0/4.0.4/4.1.3/4.2.0 class descriptors. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor (kimi-k3) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
