geoffreyclaude commented on PR #23014: URL: https://github.com/apache/datafusion/pull/23014#issuecomment-4933925240
> I am pretty sure this code is unreachable via SQL / the external APIs as small IN LISTS get rewritten to an `OR` chain (see my other comment) > > Thus if we are going to add this specialization, we should also turn off the IN LIST rewrite (and benchmark performance against the OR) The OR rewrite threshold is currently set to 3 by [THRESHOLD_INLINE_INLIST](https://github.com/apache/datafusion/blob/main/datafusion%2Foptimizer%2Fsrc%2Fsimplify_expressions%2Fexpr_simplifier.rs#L116), so this code is reachable starting at size 4. I also ran some micro benchmarks locally to compare OR rewrite to the branchless case, and branchless is faster even for size 2 (1.5x for size 2, 2.5x for size 4.) I'll look into committing these somewhere clean. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
