Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24211 )
Change subject: IMPALA-14909: Calcite planner: Fix validation for views with column list ...................................................................... IMPALA-14909: Calcite planner: Fix validation for views with column list This commit fixes a validation issue with some views that have a column list. Specifically... For the following view: create view functional.sample_view (abc) as select sum(bigint_col) from functional.alltypestiny; When this view is created, a select wrapper is placed around the view to handle the column list. In this example, the sum() column does not contain an alias. A look at the database shows the following query is stored when a "show create view" command is run on this view: CREATE VIEW functional.sample_view AS SELECT sample_view.`_c0` abc FROM (SELECT sum(bigint_col) FROM functional.alltypestiny) sample_view Note here that a '_c0' column is created in the select wrapper. But since this column does not exist in the inner select, this sql will not compile when expanded! The fix is a little hacky here, but all this view sql is out in the wild, so a hacky way is the only way to handle this. Two passes are run on validation. The first pass on the view will run the SQL as/is which will fail. Once the exception is thrown, a scan of the top layer and the inner select checks to see if there is an alias mismatch. Specifically, it will look for the _c<x> columns. If these columns are found, a replacement is done within the SqlNode tree to supply an alias that will be found. A second validation pass is done at this point. If this was the only problem in validation, the second pass should succeed. In addition to this change, the view expander was modified. The new code now ensures that validation for the view is run at validation time rather than waiting for SqlNode to RelNode conversion time. It also ensures that the SQL parsing of the view SQL only happens once for the view in a query, and the validated SqlNode is stored within the ImpalaViewTable object. The SqlNode to RelNode conversion is still handled by the RelNodeConverter, and the conversion is done through the "toRel" for the ViewTable. For testing, any query with "functional.complex_view" in the test suite did not compile previous to this fix, and now it should compile. Change-Id: I0d0229c8907f69e648034d12fe375d9d9a384e25 Reviewed-on: http://gerrit.cloudera.org:8080/24211 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M java/calcite-planner/src/main/java/org/apache/impala/calcite/rules/ImpalaMQContext.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteDb.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/ImpalaViewTable.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteAnalysisDriver.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteRelNodeConverter.java M java/calcite-planner/src/main/java/org/apache/impala/calcite/service/ImpalaSqlValidatorImpl.java A java/calcite-planner/src/main/java/org/apache/impala/calcite/service/ViewAliasCorrector.java 7 files changed, 486 insertions(+), 33 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/24211 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I0d0229c8907f69e648034d12fe375d9d9a384e25 Gerrit-Change-Number: 24211 Gerrit-PatchSet: 13 Gerrit-Owner: Steve Carlin <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Fang-Yu Rao <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jason Fehr <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Steve Carlin <[email protected]>
