GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/11283
[SPARK-12720] [SQL] SQL Generation Support for Cube, Rollup, and Grouping
Sets
## What changes were proposed in this pull request?
This PR is for supporting SQL generation for cube, rollup and grouping sets.
For example, a query using rollup:
```SQL
SELECT count(*) as cnt, key % 5, grouping_id() FROM t1 GROUP BY key % 5
WITH ROLLUP
```
Original logical plan:
```
Aggregate [(key#17L % cast(5 as bigint))#47L,grouping__id#46],
[(count(1),mode=Complete,isDistinct=false) AS cnt#43L,
(key#17L % cast(5 as bigint))#47L AS _c1#45L,
grouping__id#46 AS _c2#44]
+- Expand [List(key#17L, value#18, (key#17L % cast(5 as bigint))#47L, 0),
List(key#17L, value#18, null, 1)],
[key#17L,value#18,(key#17L % cast(5 as
bigint))#47L,grouping__id#46]
+- Project [key#17L,
value#18,
(key#17L % cast(5 as bigint)) AS (key#17L % cast(5 as
bigint))#47L]
+- Subquery t1
+- Relation[key#17L,value#18] ParquetRelation
```
Converted SQL:
```SQL
SELECT count( 1) AS `cnt`,
(`t1`.`key` % CAST(5 AS BIGINT)),
grouping_id() AS `_c2`
FROM `default`.`t1`
GROUP BY (`t1`.`key` % CAST(5 AS BIGINT))
GROUPING SETS (((`t1`.`key` % CAST(5 AS BIGINT))), ())
```
## How was the this patch tested?
Added six test cases in LogicalPlanToSQLSuite.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark groupingSetsToSQL
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11283.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11283
----
commit bc0c0309b1ec0d5303b744025032070f00c2bc9c
Author: gatorsmile <[email protected]>
Date: 2016-02-20T15:27:52Z
SQL generation support for cube, rollup, and grouping set
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
