srielau commented on PR #58553: URL: https://github.com/apache/spark/pull/58553#issuecomment-5588488063
Thanks for picking this up, and sorry for the duplicate. I independently landed the same JIRA as https://github.com/apache/spark/pull/58585 after this PR was already open. Your patch covers the same IN-list zip/NULL-order and multi-field struct padding bugs, and it goes further in two places I think are the right design: - Fixing the redundant same-type `Cast` in `InTypeCoercion` so `ApplyCharTypePadding` still sees the CHAR attribute, rather than peeling analyzer casts later. - Skipping padding on non-orderable types so `CheckAnalysis` still names the user's expression. I will close #58585 as a duplicate in favor of this PR. One small extra coverage item from #58585, if you want it: with `spark.sql.optimizer.inSetConversionThreshold=1`, `c IN (NULL, 'a', 'b')` should still be `true` and `c IN (NULL, 'x', 'y')` should still be `null`. `OptimizeIn` can freeze a corrupted list into `InSet`, so that path is worth locking in. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
