[
https://issues.apache.org/jira/browse/SPARK-36819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418301#comment-17418301
]
Apache Spark commented on SPARK-36819:
--------------------------------------
User 'Swinky' has created a pull request for this issue:
https://github.com/apache/spark/pull/34062
> DPP: Don't insert redundant filters in case static partition pruning can be
> done
> --------------------------------------------------------------------------------
>
> Key: SPARK-36819
> URL: https://issues.apache.org/jira/browse/SPARK-36819
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.1.2
> Reporter: Swinky Mann
> Priority: Minor
>
> Don't insert dynamic partition pruning filters in case the filters already
> referred statically. In case the filtering predicate on dimension table is in
> joinKey, no need to insert DPP filter in that case.
> DPP is not required in this Sample query:
> {{SELECT f.date_id, f.pid, f.sid FROM
> (select date_id, product_id as pid, store_id as sid from fact_stats) as f
> JOIN dim_stats s
> ON f.sid = s.store_id WHERE s.store_id = 3}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
