mgaido91 commented on issue #23171: [SPARK-26205][SQL] Optimize In for bytes, shorts, ints URL: https://github.com/apache/spark/pull/23171#issuecomment-446916902 thanks @aokolnychyi. I just have a couple of comments on this: 1 - As @rxin mentioned, now we have InSet for handling Literals and In for handling arbitrary expressions. Since this method works only with literals, I'd rather see it as an alternative execution for InSet rather than for In. Then we might want to convert always (without a threshold) a In containing literals in InSet and let InSet pick the best implementation (either switch or the real InSet). What do you think about this? 2 - I think we may also support longs. We just need to split a long in 2 integers, so with 2 nested switches it would be doable I think. I see this will add complexity and we need to write a dedicated implementation for it, but we may consider this as a followup work. Do you agree on this?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
