GitHub user skambha opened a pull request:
https://github.com/apache/spark/pull/23206
[SPARK-26249][SQL] Add ability to inject a rule in order and to add a batch
via the Spark Extension Points API
## What changes were proposed in this pull request?
Add two new APIs to SparkSessionExtensions:
- Inject optimizer rule in a specific order
- Inject optimizer batch.
Design details is here:
https://drive.google.com/file/d/1m7rQZ9OZFl0MH5KS12CiIg3upLJSYfsA/view?usp=sharing
## How was this patch tested?
- New unit tests have been added to the SparkSessionExtensionSuite
- Ran the sql, hive, catalyst unit tests without any issues.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/skambha/spark ext1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/23206.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #23206
----
commit b25c31666c0136e3d98962b51a8637a21e63278a
Author: Sunitha Kambhampati <skambha@...>
Date: 2018-11-30T11:42:59Z
Add 2 api to the extension points and add unit tests
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
