Impala Public Jenkins has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/24089 )
Change subject: IMPALA-14850: Codegen tuple DeepCopy for hash join
......................................................................
IMPALA-14850: Codegen tuple DeepCopy for hash join
Created codegen'd version of BufferedTupleStream::DeepCopy and
BufferedTupleStream::AddRow.
Codegen'd function is only used by PartitionedHashJoinBuilder in this
patch.
This patch does not do proper codegen for collection types, instead it
calls the interpreted code for them.
It was considered to use Tuple's TryDeepCopy* functions for
BufferedTupleStream, but it's better to keep its own DeepCopy
for there are differences between the two:
-BufferedTupleStream doesn't copy tuples serially, first
it copies "fixed len" parts of all tuples, then all
"string data" for all tuples, then all "collection data" of
all tuples.
-BufferedTupleStream's DeepCopy doesn't set String's pointers.
This also applies when copying a string from a collection.
Measurements:
Measured with the following commit:
select straight_join l_orderkey, o_custkey, o_orderkey, l_partkey
from tpch30.orders left join /*+broadcast*/ tpch30.lineitem
on o_orderkey = l_orderkey where o_totalprice<0;
Where tpch30 is generated by:
bin/load-data.py -s 30 -f --workloads tpch
--table_formats text/none,parquet/snap
Before:
BuildRowsPartitionTime: 3s996ms
After:
BuildRowsPartitionTime: 2s139ms
Codegen:
Before:
NumInstructions: 7.04K (7040)
NumOptimizedInstructions: 2.66K (2664)
ModuleBitcodeSize: 3.43 MB (3592912)
After:
NumInstructions: 7.11K (7111)
NumOptimizedInstructions: 2.72K (2719)
ModuleBitcodeSize: 3.43 MB (3596392)
Testing:
Changed tests in buffered-tuple-stream-test to use both interpreted
and codegened versions of BufferedTupleStream::AddRow.
Change-Id: I63e32babdbaf56095478c6c66afb9cb91189f946
Reviewed-on: http://gerrit.cloudera.org:8080/24089
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
---
M be/src/codegen/gen_ir_descriptions.py
M be/src/codegen/impala-ir.cc
M be/src/exec/partitioned-hash-join-builder-ir.cc
M be/src/exec/partitioned-hash-join-builder.cc
M be/src/exec/partitioned-hash-join-builder.h
A be/src/exec/partitioned-hash-join-builder.inline.h
M be/src/runtime/CMakeLists.txt
A be/src/runtime/buffered-tuple-stream-ir.cc
M be/src/runtime/buffered-tuple-stream-test.cc
M be/src/runtime/buffered-tuple-stream.cc
M be/src/runtime/buffered-tuple-stream.h
M be/src/runtime/spillable-row-batch-queue.h
A be/src/runtime/tuple-row-ir.cc
M be/src/runtime/tuple-row.h
14 files changed, 749 insertions(+), 239 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/24089
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I63e32babdbaf56095478c6c66afb9cb91189f946
Gerrit-Change-Number: 24089
Gerrit-PatchSet: 25
Gerrit-Owner: Balazs Hevele <[email protected]>
Gerrit-Reviewer: Balazs Hevele <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Yida Wu <[email protected]>
