[
https://issues.apache.org/jira/browse/IMPALA-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18094698#comment-18094698
]
ASF subversion and git services commented on IMPALA-14850:
----------------------------------------------------------
Commit 621a55df7d0dc790ce1e7f6aa4ba0966615bcdb6 in impala's branch
refs/heads/master from Balazs Hevele
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=621a55df7 ]
IMPALA-14850: Codegen tuple DeepCopy for hash join
Created codegen'd version of BufferedTupleStream::DeepCopy and
BufferedTupleStream::AddRow.
Codegen'd function is only used by PartitionedHashJoinBuilder in this
patch.
This patch does not do proper codegen for collection types, instead it
calls the interpreted code for them.
It was considered to use Tuple's TryDeepCopy* functions for
BufferedTupleStream, but it's better to keep its own DeepCopy
for there are differences between the two:
-BufferedTupleStream doesn't copy tuples serially, first
it copies "fixed len" parts of all tuples, then all
"string data" for all tuples, then all "collection data" of
all tuples.
-BufferedTupleStream's DeepCopy doesn't set String's pointers.
This also applies when copying a string from a collection.
Measurements:
Measured with the following commit:
select straight_join l_orderkey, o_custkey, o_orderkey, l_partkey
from tpch30.orders left join /*+broadcast*/ tpch30.lineitem
on o_orderkey = l_orderkey where o_totalprice<0;
Where tpch30 is generated by:
bin/load-data.py -s 30 -f --workloads tpch
--table_formats text/none,parquet/snap
Before:
BuildRowsPartitionTime: 3s996ms
After:
BuildRowsPartitionTime: 2s139ms
Codegen:
Before:
NumInstructions: 7.04K (7040)
NumOptimizedInstructions: 2.66K (2664)
ModuleBitcodeSize: 3.43 MB (3592912)
After:
NumInstructions: 7.11K (7111)
NumOptimizedInstructions: 2.72K (2719)
ModuleBitcodeSize: 3.43 MB (3596392)
Testing:
Changed tests in buffered-tuple-stream-test to use both interpreted
and codegened versions of BufferedTupleStream::AddRow.
Change-Id: I63e32babdbaf56095478c6c66afb9cb91189f946
Reviewed-on: http://gerrit.cloudera.org:8080/24089
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Codegen tuple copy for Hash Join
> --------------------------------
>
> Key: IMPALA-14850
> URL: https://issues.apache.org/jira/browse/IMPALA-14850
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Balazs Hevele
> Assignee: Balazs Hevele
> Priority: Major
> Labels: codegen, performance
>
> Add codegen for BufferedTupleStream::DeepCopy, and use it for PhjBuilder.
> If possible, use common deepcopy functions with other systems (e.g.
> Tuple::TryDeepCopy*).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
