GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/20201
[SPARK-22389][SQL] data source v2 partitioning reporting interface
## What changes were proposed in this pull request?
a new interface which allows data source to report partitioning and avoid
shuffle at Spark side.
The design is pretty like the internal distribution/partitioing framework.
Spark defines a `Distribution` interfaces and several concrete implementations,
and ask the data source to report a `Partitioning`, the `Partitioning` should
tell Spark if it can satisfy a `Distribution` or not.
## How was this patch tested?
new test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark partition-reporting
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20201.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20201
----
commit be14e3bd7598eb3ed583e18c1d9927d5c7f563b4
Author: Wenchen Fan <wenchen@...>
Date: 2018-01-09T02:08:53Z
data source v2 partitioning reporting interface
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
