Malthe Borch created SPARK-55569:
------------------------------------
Summary: Filter pushdown in pivot aggregation
Key: SPARK-55569
URL: https://issues.apache.org/jira/browse/SPARK-55569
Project: Spark
Issue Type: Wish
Components: Optimizer
Affects Versions: 4.1.1
Reporter: Malthe Borch
When pivoting on a set of values, this is potentially a subset of all values.
Nonetheless, there's no predicate pushdown to the source.
The workaround is to repeat the same set of values in a WHERE-clause before the
pivot clause, but this is redundant. In addition, I think it can be argued that
you expect the optimizer to do this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
