[
https://issues.apache.org/jira/browse/IMPALA-14852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18099894#comment-18099894
]
ASF subversion and git services commented on IMPALA-14852:
----------------------------------------------------------
Commit 87768d97bc291f1eb4184a476d7b271b741a074c in impala's branch
refs/heads/master from Balazs Hevele
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=87768d97b ]
IMPALA-14852 Codegen tuple TryDeepCopy for Broadcast Exchange
Use Tuple::TryDeepCopy in RowBatch/OutboundRowBatch instead of DeepCopy.
For most tuples, this should save an extra size calculation, because the
size of the whole row is not calculated anymore before the copying, only
if a tuple doesn't fit, is its size calculated.
A codegen'd version of Tuple::TryDeepCopy is used for each tuple in the
row, which is generated based on the Tuple's Descriptor.
This patch does not do proper codegen for collection types, instead it
calls the interpreted code for them.
Measurements:
Measured with query:
SELECT l1.* FROM tpch30.lineitem l1
INNER JOIN tpch30.lineitem l2
ON l1.l_shipmode = l2.l_shipmode LIMIT 1;
Where tpch30 is generated by:
bin/load-data.py -s 30 -f --workloads tpch
--table_formats text/none,parquet/snap
SerializeBatchTime changed from 1s797ms to 1s239ms.
This is about 31% improvement.
Codegen:
Before:
NumInstructions: 4.01K (4015)
NumOptimizedInstructions: 3.87K (3874)
ModuleBitcodeSize: 3.68 MB (3860700)
After:
NumInstructions: 5.26K (5259)
NumOptimizedInstructions: 5.15K (5151)
ModuleBitcodeSize: 3.71 MB (3892344)
Change-Id: Iaa43e949c63db047717104dbbe42d47f94ebf2d0
Reviewed-on: http://gerrit.cloudera.org:8080/24154
Reviewed-by: Csaba Ringhofer <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Codegen tuple copy for Broadcast Exchange
> -----------------------------------------
>
> Key: IMPALA-14852
> URL: https://issues.apache.org/jira/browse/IMPALA-14852
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Balazs Hevele
> Priority: Major
>
> Codegen tuple copy for OutBoundRowBatch, and use it in
> KrpcDataStreamRecvr/Sender.
> If possible, use common deepcopy functions with other systems (e.g.
> Tuple::TryDeepCopy*).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
