Csaba Ringhofer has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19879
Change subject: IMPALA-12138: Optimize HS2 result vector allocations ...................................................................... IMPALA-12138: Optimize HS2 result vector allocations Before this patch the reservation sizes were based on the number of rows in the RowBatches - as batch_size has lower default than fetch_size (1024 vs 10240, one fetch is served by multiple row batches leading to reserving vectors in more than one step. This patch changes the logic to: - reserve during the first fetch the old way - reserve fetch_size in subsequent fetches This means that queries with small result set should not regress while in large ones only the first and the last fetches will be suboptimal. Also noticed that the current default fetch_size=10240 in impala-shell is not optimal for RowMaterializationTimer, probably because it is not power of 2 and leads to overallocation. Tested with select * from tpch_parquet.lineitem, and RowMaterializationTimer was decreased around 10-20%: fetch_size=10240: 3.6s -> 3.2s fetch_size=8192: 2.8s->2.6s Change-Id: I7b0e6a0a8fd028e3c0e4f1f4e272a50d2bfb59ba --- M be/src/service/hs2-util.cc M be/src/service/hs2-util.h M be/src/service/impala-hs2-server.cc M be/src/service/query-result-set.cc M be/src/service/query-result-set.h 5 files changed, 49 insertions(+), 31 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/79/19879/1 -- To view, visit http://gerrit.cloudera.org:8080/19879 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I7b0e6a0a8fd028e3c0e4f1f4e272a50d2bfb59ba Gerrit-Change-Number: 19879 Gerrit-PatchSet: 1 Gerrit-Owner: Csaba Ringhofer <[email protected]>
