Hello Aman Sinha, Quanlong Huang, Fang-Yu Rao, Jason Fehr, Joe McDonnell,
Michael Smith, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24211
to look at the new patch set (#6).
Change subject: IMPALA-14909: Calcite planner: Fix validation for views with
column list
......................................................................
IMPALA-14909: Calcite planner: Fix validation for views with column list
This commit fixes a validation issue with some views that have a column
list.
Specifically...
For the following view:
create view functional.sample_view (abc) as
select sum(bigint_col) from functional.alltypestiny;
When this view is created, a select wrapper is placed around the view
to handle the column list. In this example, the sum() column does not
contain an alias. A look at the database shows the following query is
stored when a "show create view" command is run on this view:
CREATE VIEW functional.sample_view AS
SELECT sample_view.`_c0` abc FROM (SELECT sum(bigint_col)
FROM functional.alltypestiny) sample_view
Note here that a '_c0' column is created in the select wrapper. But
since this column does not exist in the inner select, this sql will
not compile when expanded!
The fix is a little hacky here, but all this view sql is out in the
wild, so a hacky way is the only way to handle this.
Two passes are run on validation. The first pass on the view will run
the SQL as/is which will fail. Once the exception is thrown, a scan
of the top layer and the inner select checks to see if there is an
alias mismatch. Specifically, it will look for the _c<x> columns. If
these columns are found, a replacement is done within the SqlNode tree
to supply an alias that will be found. A second validation pass is done
at this point. If this was the only problem in validation, the second
pass should succeed.
In addition to this change, the view expander was modified. The new
code now ensures that validation for the view is run at validation
time rather than waiting for SqlNode to RelNode conversion time. It also
ensures that the SQL parsing of the view SQL only happens once for the
view in a query, and the validated SqlNode is stored within the
ImpalaViewTable object.
The SqlNode to RelNode conversion is still handled by the RelNodeConverter,
and the conversion is done through the "toRel" for the ViewTable.
For testing, any query with "functional.complex_view" in the test suite
did not compile previous to this fix, and now it should compile.
Change-Id: I0d0229c8907f69e648034d12fe375d9d9a384e25
---
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/ImpalaMQContext.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteDb.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/ImpalaViewTable.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteAnalysisDriver.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteRelNodeConverter.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/ImpalaSqlValidatorImpl.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/ViewAliasCorrector.java
7 files changed, 487 insertions(+), 30 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/11/24211/6
--
To view, visit http://gerrit.cloudera.org:8080/24211
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0d0229c8907f69e648034d12fe375d9d9a384e25
Gerrit-Change-Number: 24211
Gerrit-PatchSet: 6
Gerrit-Owner: Steve Carlin <[email protected]>
Gerrit-Reviewer: Aman Sinha <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Fang-Yu Rao <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Jason Fehr <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Steve Carlin <[email protected]>
