shivadarshan-devadiga commented on PR #58553: URL: https://github.com/apache/spark/pull/58553#issuecomment-5589622021
Thanks @srielau for closing [#58585](https://github.com/apache/spark/pull/58585) in favour of this one and for the review of the design choices. I've taken your coverage suggestion and added it to the test case: - With spark.sql.optimizer.inSetConversionThreshold=1, c IN (null, 'a', 'b') is true and c IN (null, 'x', 'y') is null. The block also asserts the optimized plan really contains an InSet, since OptimizeIn only converts when every list element is a Literal (In.inSetConvertible) — which here depends on ConstantFolding collapsing the coercion Cast and the padding rpad first. Without that assert, a change in folding order could quietly put the case back on the plain In path while still passing. - A collated variant in the same block, since InSet routes non-UTF8-binary collations through CollationAwareSet, which keys on the collation key rather than binary UTF8String equality. Padding + collation + set lookup wasn't covered anywhere else. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
