GitHub user skambha opened a pull request:
https://github.com/apache/spark/pull/11480
[SPARK-13630][SQL] Adds optimizer rule collapsesorts to collapse adjaâ¦
## What changes were proposed in this pull request?
This patch does the following:
I) Adds a new optimizer rule collapsesorts that does the following if
global is same for the adjacent sorts.
a) Collapse adjacent sorts and keep the last sort
b) Collapse adjacent sorts if there is a project or a limit or a filter in
between and keep the last sort.
II) A new test suite CollapseSortsSuite is added with tests.
Also note, one of the _testcase (test("collapsesorts: test collapsesorts in
sort <- limit <- sort scenario") ) _does not compare with expected plan because
of the unapply in Limit will actually remove the LocalLimit from the plan.
Hence the test just checks that the collapsesorts rule was exercised by
checking for the number of Sort in the plan.
## How was this patch tested?
A)
Following test suites were run and the lint checking was done. No new test
failures:
build/sbt -Phive hive/test
build/sbt sql/test
build/sbt catalyst/test
dev/lint-scala
B) A new test suite CollapseSortsSuite is added with new tests to exercise
the collapsesorts rule.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/skambha/spark SPARK-13630
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11480.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11480
----
commit 7e2450efbf7b33cc33bb0b2946ba790cf27e1bac
Author: Sunitha Kambhampati <[email protected]>
Date: 2016-03-02T23:59:38Z
[SPARK-13630][SQL] Adds optimizer rule collapsesorts to collapse adjacent
sorts and add new testsuite
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
