[
https://issues.apache.org/jira/browse/IMPALA-12872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17840957#comment-17840957
]
ASF subversion and git services commented on IMPALA-12872:
----------------------------------------------------------
Commit b39cd79ae84c415e0aebec2c2b4d7690d2a0cc7a in impala's branch
refs/heads/master from Steve Carlin
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b39cd79ae ]
IMPALA-12872: Use Calcite for optimization - part 1: simple queries
This is the first commit to use the Calcite library to parse,
analyze, and optimize queries.
The hook for the planner is through an override of the JniFrontend. The
CalciteJniFrontend class is the driver that walks through each of the
Calcite steps which are as follows:
CalciteQueryParser: Takes the string query and outputs an AST in the
form of Calcite's SqlNode object.
CalciteMetadataHandler: Iterate through the SqlNode from the previous step
and make sure all essential table metadata is retrieved from catalogd.
CalciteValidator: Validate the SqlNode tree, akin to the Impala Analyzer.
CalciteRelNodeConverter: Change the AST into a logical plan. In this first
commit, the only logical nodes used are LogicalTableScan and LogicalProject.
The LogicalTableScan will serve as the node that reads from an Hdfs Table and
the LogicalProject will only project out the used columns in the query. In
later versions, the LogicalProject will also handle function changes.
CalciteOptimizer: This step is to optimize the query. In this cut, it will be
a nop, but in later versions, it will perform logical optimizations via
Calcite's rule mechanism.
CalcitePhysPlanCreator: Converts the Calcite RelNode logical tree into
Impala's PlanNode physical tree
ExecRequestCreator: Implement the existing Impala steps that turn a Single
Node Plan into a Distributed Plan. It will also create the TExecRequest object
needed by the runtime server.
Only some very basic queries will work with this commit. These include:
select * from tbl <-- only needs the LogicalTableScan
select c1 from tbl <-- Also uses the LogicalProject
In the CalciteJniFrontend, there is some basic checks to make sure only
select statements will get processed. Any non-query statement will revert
back to the current Impala planner.
In this iteration, any queries besides the minimal ones listed above will
result in a caught exception which will then be run through the current
Impala planner. The tests that do work can be found in calcite.test and
run through the custom cluster test test_experimental_planner.py
This iteration should support all types with the exception of complex
types. Calcite does not have a STRING type, so the string type is
represented as VARCHAR(MAXINT) similar to how Hive represents their
STRING type.
The ImpalaTypeConverter file is used to convert the Impala Type object
to corresponding Calcite objects.
Authorization is not yet working with this current commit. A Jira has been
filed (IMPALA-13011) to deal with this.
Change-Id: I453fd75b7b705f4d7de1ed73c3e24cafad0b8c98
Reviewed-on: http://gerrit.cloudera.org:8080/21109
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Reviewed-by: Joe McDonnell <joemcdonn...@cloudera.com>
> Support basic queries using Calcite on the frontend
> ---------------------------------------------------
>
> Key: IMPALA-12872
> URL: https://issues.apache.org/jira/browse/IMPALA-12872
> Project: IMPALA
> Issue Type: Sub-task
> Components: fe
> Reporter: Steve Carlin
> Priority: Major
>
> First commit for the Calcite planner. The idea is to make minimal changes to
> the current frontend code and keep this code in a separate jar.
>
> The first commit will support a basic select statement without any filters or
> functions (e.g. select * from tbl and select c1 from tbl)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
