[
https://issues.apache.org/jira/browse/IMPALA-12138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17723080#comment-17723080
]
ASF subversion and git services commented on IMPALA-12138:
----------------------------------------------------------
Commit 3dfebca9b16d7cf4ced40f7efac5d05ac5fe51d9 in impala's branch
refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=3dfebca9b ]
IMPALA-12138: Optimize HS2 result vector allocations
Before this patch the reservation sizes were based on the
number of rows in the RowBatches - as batch_size has lower default
than fetch_size (1024 vs 10240), one fetch is served by multiple row
batches leading to reserving vectors in more than one step.
This patch changes the logic to:
- reserve during the first fetch the old way
- reserve fetch_size in subsequent fetches
This means that queries with small result set should not regress
while in large ones only the first and the last fetches will be
suboptimal.
Also noticed that the current default fetch_size=10240 in impala-shell
is not optimal for RowMaterializationTimer, probably because it is
not a power of 2 and leads to overallocation.
Created IMPALA-12142 for the potential default fetch_size change.
Tested with select * from tpch_parquet.lineitem, and
RowMaterializationTimer was decreased around 10-20%:
fetch_size=10240: 3.6s -> 3.2s
fetch_size=8192: 2.8s -> 2.6s
Change-Id: I7b0e6a0a8fd028e3c0e4f1f4e272a50d2bfb59ba
Reviewed-on: http://gerrit.cloudera.org:8080/19879
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Suboptimal vector allocation of HS2 results
> -------------------------------------------
>
> Key: IMPALA-12138
> URL: https://issues.apache.org/jira/browse/IMPALA-12138
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Daniel Becker
> Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
