GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/11635
[SPARK-13739] [SQL] [WIP] Push Predicate Through Window
#### What changes were proposed in this pull request?
For performance, predicates can be pushed through Window if and only if the
following conditions are satisfied:
- Predicate involves one and only one column that is part of window
partitioning key
- Window partitioning key is just a sequence of attributeReferences. (i.e.,
none of them is an expression)
- Predicate must be deterministic
#### How was this patch tested?
TODO: more testing will be added. DSL needs to be modified for window
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark pushPredicateThroughWindow
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11635.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11635
----
commit 458f7be8f88cb83e348b72fe9350bfa9c12282a4
Author: gatorsmile <[email protected]>
Date: 2016-03-10T18:26:36Z
ppd for window
commit 92136ddc73830f7af98afff17cb05d939c0ad776
Author: gatorsmile <[email protected]>
Date: 2016-03-10T18:33:26Z
Merge remote-tracking branch 'upstream/master' into
pushPredicateThroughWindow
commit f401d8bdc97ac163ff5fc3d0a75344a8b867414e
Author: gatorsmile <[email protected]>
Date: 2016-03-10T18:47:02Z
only partitioning key
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
