Yash Datta created SPARK-3711:
---------------------------------
Summary: Optimize where in clause filter queries
Key: SPARK-3711
URL: https://issues.apache.org/jira/browse/SPARK-3711
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 1.1.0
Reporter: Yash Datta
Priority: Minor
Fix For: 1.1.1
The In case class is replaced by a InSet class in case all the filters are
literals, which uses a hashset instead of Sequence, thereby giving significant
performance improvement. Maximum improvement should be visible in case small
percentage of large data matches the filter list
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
