GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/11815
[SPARK-12789] [SQL] Support Order By Ordinal in SQL
#### What changes were proposed in this pull request?
This PR is to support order by position in SQL, e.g.
```SQL
select c1, c2, c3 from tbl order by 1 desc, 3
```
should be equivalent to
```SQL
select c1, c2, c3 from tbl order by c1 desc, c3 asc
```
This is controlled by config option `spark.sql.orderByOrdinal`.
- When true, the ordinal number is treated as the position in the select
list.
- When false, the ordinal number in Order/Sort By clause is not allowed. If
users put the sort by column without any reference, users will get an exception
with a meaningful message.
- Only convert integer literals (not foldable expressions). If found
foldable expressions, issue an exception with a meaningful message.
- This also works with select *.
**Question**: Do we still need sort by columns that contain zero reference?
In this case, it will have no impact on the sorting results. IMO, we should not
allow users do it. @rxin @cloud-fan @marmbrus @yhuai @hvanhovell
This PR is taken from https://github.com/apache/spark/pull/10731. Please
give the credit to @zhichao-li
Also cc all the people who are involved in the previous discussion:
@adrian-wang @chenghao-intel @tejasapatil
#### How was this patch tested?
Added a few test cases for both positive and negative test cases.
Removed two cases in which order by clauses have the columns that have no
impact on the sorting results.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark orderByPosition
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11815.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11815
----
commit aee1023abd8e7d0e0d0cdc3c21c4d500522926c2
Author: gatorsmile <[email protected]>
Date: 2016-03-18T06:15:54Z
Order by ordinal number.
commit 33427981564f98e2fb70720b3eced9c37063a181
Author: gatorsmile <[email protected]>
Date: 2016-03-18T06:20:38Z
Merge remote-tracking branch 'upstream/master' into orderByPosition
commit 240aeb3c1d099ebcc4321cbf0987aeccc8e1cbd6
Author: gatorsmile <[email protected]>
Date: 2016-03-18T06:21:03Z
style fix.
commit e0dce39806fe4c9e9bec9f85727456820a990b99
Author: gatorsmile <[email protected]>
Date: 2016-03-18T06:42:26Z
improve the test cases.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
