mgaido91 commented on issue #23171: [SPARK-26205][SQL] Optimize In for bytes, shorts, ints URL: https://github.com/apache/spark/pull/23171#issuecomment-447559881 we can keep `spark.sql.optimizer.inSetConversionThreshold` as of now, just read it in `InSet` and behave accordingly. Then, we can tune/remove this config according to the results of the benchmarks. > certain data types will become slightly slower those datatypes are rather uncommon I'd say, apart from struct. And the difference was very small IIRC. So it may be not a big problem if this allows a cleaner design I think.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
