[
https://issues.apache.org/jira/browse/IMPALA-6501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956464#comment-16956464
]
ASF subversion and git services commented on IMPALA-6501:
---------------------------------------------------------
Commit 3f1d0472243f25eacf611b12a5e1150294683957 in impala's branch
refs/heads/master from norbert.luksa
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=3f1d047 ]
IMPALA-6501: Optimize count(star) for Kudu scans
IMPALA-5036 added an optimisation for count(star) in Parquet scans
that avoids materialising dummy rows. This change provides similar
optimization for Kudu tables.
Instead of materializing empty rows when computing count star, we use
the NumRows field from the Kudu API. The Kudu scanner tuple is
modified to have one slot into which we will write the
num rows statistic. The aggregate function is changed from count to a
special sum function that gets initialized to 0.
Tests:
* Added end-to-end tests
̣* Added planner tests
* Run performance tests on tpch.lineitem Kudu table with 25 set as
scaling factor, on 1 node, with mt_dop set to 1, just to measure
the speedup gained when scanning. Counting the rows before the
optimization took around 400ms, and around 170ms after.
Change-Id: Ic99e0f954d0ca65779bd531ca79ace1fcb066fb9
Reviewed-on: http://gerrit.cloudera.org:8080/14347
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Optimize count(*) for Kudu scans
> --------------------------------
>
> Key: IMPALA-6501
> URL: https://issues.apache.org/jira/browse/IMPALA-6501
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.11.0
> Reporter: Tim Armstrong
> Assignee: Norbert Luksa
> Priority: Major
> Labels: kudu, ramp-up
>
> IMPALA-5036 added an optimisation for count(\*) in Parquet scans that avoids
> materialising dummy rows. We should do something similar for Kudu.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
