[
https://issues.apache.org/jira/browse/IMPALA-14815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18099322#comment-18099322
]
ASF subversion and git services commented on IMPALA-14815:
----------------------------------------------------------
Commit 08c8cf9b7f39c0fb057b11dfcb5f8d07fcb0ea12 in impala's branch
refs/heads/master from Steve Carlin
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=08c8cf9b7 ]
IMPALA-15019: Avoid materializing non-projected filtered partition column.
This commit is a small optimization to the Calcite planner that already
exists in the original planner. For a query like:
select id from functional_parquet.alltypes where year = 2010
The year column is partitioned. Because of this, there is no reason to
materialize the column in the scan node. The columns where year is not
2010 are contained within the partitioned directory and are pruned out.
Since the year column is not selected, it does not need to show up
anywhere as the data is selected.
To enable this, a "filterOnlyInputRefs" field was added to the context
that is passed down through the RelNodes at physical creation time. On
the Filter RelNode, it adds any input ref field in the filter condition
that is not projected out through its parent Project RelNode.
In order for this to work, it is worth mentioning that it relies on the
fact that the only way a Project can exist between a Filter and a
TableScan (i.e. the Filter-Project-TableScan hierarchy) is when the
RelFieldTrimmer adds the Project that essentially "trims" the fields
coming out of the scan immediately after the scan occurs. Any Project
that manipulates fields will be on top of the Filter via the
FilterProjectTranspose rule, so the Project underneath the Filter will
only contain non-manipulated RexInputRefs as its project fields.
IMPALA-14815 is a duplicate of this Jira, and some tests have been fixed
in the stats-extrapolation.test file.
Change-Id: I902dc625963321aff57e5c05af3d1137c06aa882
Reviewed-on: http://gerrit.cloudera.org:8080/24558
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Calcite planner: Unneeded filter field in output expr for scan node
> -------------------------------------------------------------------
>
> Key: IMPALA-14815
> URL: https://issues.apache.org/jira/browse/IMPALA-14815
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Steve Carlin
> Priority: Major
>
> In stats-extrapolation.test for the query:
> explain select id from alltypes where month in (1, 2, 3);
> The row-size for the scan node is 8B for Calcite. The month field does not
> need to be in the output expressions.
> Note, on a preliminary investigation, it seems that the filter field is not
> in the output exprs all the time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
