Quanlong Huang has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/24105 )
Change subject: IMPALA-2083: Add PIVOT clause
......................................................................
IMPALA-2083: Add PIVOT clause
This patch adds support for the PIVOT clause by adding a new subclass
of `TableRef` called `PivotTableRef`. The concepts and the examples are
explained in `PivotTableRef.java`.
The changes are mostly in the frontend:
- A new keyword `PIVOT` is added.
- In the parsing phase, a PIVOT clause will be parsed into a
`PivotTableRef`.
- Then, in the analysis phase, the `PivotTableRef` will be rewritten to
an `InlineViewRef` to a nested subquery, in which
- First, the source table will be aggregated with the aggregate
expressions specified in the PIVOT clause;
- Then, each aggregated column will be split into multiple columns
with the `aggif()` function, one for each value in the PIVOT clause.
No new plan node or new execution node is added.
Since it is now possible that the predicate in the `aggif()` function
references a nullable slot, this patch removes the restriction that the
predicate must not be NULL.
Limitations:
- Multiple header columns in the PIVOT clause is not supported.
- Specifying a nested field as the header column is not supported.
- Specifying constant expressions other than literals in the header
value list is not supported.
- Using the PIVOT clause in statements other than SELECT is not
supported.
Testing:
- Added E2E tests in test_pivot_clause.py
- Added FE tests to ensure that predicates can be pushed down to the
SCAN nodes.
Change-Id: Ib799b772be18d2a3db8983b24e1068e7dfdf1ca9
Reviewed-on: http://gerrit.cloudera.org:8080/24105
Reviewed-by: Quanlong Huang <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
---
M be/src/exprs/aggregate-functions-ir.cc
M be/src/exprs/aggregate-functions.h
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/FromClause.java
A fe/src/main/java/org/apache/impala/analysis/PivotTableRef.java
M fe/src/main/java/org/apache/impala/analysis/StmtRewriter.java
M fe/src/main/java/org/apache/impala/analysis/TableRef.java
M fe/src/main/java/org/apache/impala/catalog/BuiltinsDb.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
A testdata/workloads/functional-planner/queries/PlannerTest/pivot-clause.test
A testdata/workloads/functional-query/queries/QueryTest/pivot-clause.test
M
testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking_and_row_filtering.test
A tests/query_test/test_pivot_clause.py
18 files changed, 1,194 insertions(+), 25 deletions(-)
Approvals:
Quanlong Huang: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/24105
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib799b772be18d2a3db8983b24e1068e7dfdf1ca9
Gerrit-Change-Number: 24105
Gerrit-PatchSet: 15
Gerrit-Owner: Xuebin Su <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Xuebin Su <[email protected]>
