Swinky Mann created SPARK-36819:
-----------------------------------
Summary: DPP: Don't insert redundant filters in case static
partition pruning can be done
Key: SPARK-36819
URL: https://issues.apache.org/jira/browse/SPARK-36819
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.1.2
Reporter: Swinky Mann
Don't insert dynamic partition pruning filters in case the filters already
referred statically. In case the filtering predicate on dimension table is in
joinKey, no need to insert DPP filter in that case.
DPP is not required in this Sample query:
{{SELECT f.date_id, f.pid, f.sid FROM
(select date_id, product_id as pid, store_id as sid from fact_stats) as f
JOIN dim_stats s
ON f.sid = s.store_id WHERE s.store_id = 3}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
