Sameer Agarwal created SPARK-13869:
--------------------------------------
Summary: Remove redundant conditions while combining filters
Key: SPARK-13869
URL: https://issues.apache.org/jira/browse/SPARK-13869
Project: Spark
Issue Type: Improvement
Components: SQL
Reporter: Sameer Agarwal
Priority: Minor
We should modify the existing CombineFilters rule in catalyst to remove
redundant conditions while combining individual filter predicates. For
instance, queries of the form table.where('a === 1 && 'b === 1).where('a === 1
&& 'c === 1) should be correctly optimized to table.where('a === 1 && 'b === 1
&& 'c === 1) (instead of table.where('a === 1 && 'a === 1 && 'b === 1 && 'c ===
1))
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
