[
https://issues.apache.org/jira/browse/SPARK-55569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18120936#comment-18120936
]
Shrirang Mhalgi commented on SPARK-55569:
-----------------------------------------
I would like to work on this one - will implement the feature and raise a PR
soon
> Filter pushdown in pivot aggregation
> ------------------------------------
>
> Key: SPARK-55569
> URL: https://issues.apache.org/jira/browse/SPARK-55569
> Project: Spark
> Issue Type: Wish
> Components: Optimizer
> Affects Versions: 4.1.1
> Reporter: Malthe Borch
> Priority: Major
>
> When pivoting on a set of values, this is potentially a subset of all values.
> Nonetheless, there's no predicate pushdown to the source.
> {code:sql}
> SELECT * FROM person
> PIVOT (
> SUM(age) AS a, AVG(class) AS c
> FOR name IN ('John' AS john, 'Mike' AS mike)
> );
> {code}
> In this example there is no predicate pushdown to filter the {{person}} table
> on the specified names.
> The workaround is to repeat the same set of values in a WHERE-clause before
> the pivot clause, but this is redundant. In addition, I think it can be
> argued that you expect the optimizer to do this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
