GitHub user maropu opened a pull request:
https://github.com/apache/spark/pull/10631
[SPARK-12686][SQL] Support group-by push down to data sources
This pr enables pushed-down MIN/MAX aggregation into JDBC data sources. As
for logical plan nodes like 'Aggregate -> Project -> (Filter) -> Scan', try to
push down partial aggregation processing into data sources that could aggregate
their own data efficiently because Orc/Parquet could fetch the MIN/MAX value by
using statistics data and some databases have efficient aggregation
implementations.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/maropu/spark SupportPreAggregateInDS
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/10631.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10631
----
commit ec53a07a74594c8862ee3f8a0ed8f72c5078322f
Author: Takeshi YAMAMURO <[email protected]>
Date: 2016-01-06T10:19:19Z
Support group-by push down to data sources
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
