Csaba Ringhofer has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20719 )
Change subject: IMPALA-12433: Share buffers among channels in KrpcDataStreamSender ...................................................................... IMPALA-12433: Share buffers among channels in KrpcDataStreamSender Before this patch each KrpcDataStreamSender::Channel had 2 OutboundRowBatch with its own serialization and compression buffers. This patch switches to use a single buffer per channel. This is enough to store the in-flight data in KRPC, while other buffers are only used during serialization and compression which is done for just a single channel at a time, so can be shared among channels. Memory estimates in the planner are not changed because the existing calculation has several issues (see IMPALA-12594). Change-Id: I64854a350a9dae8bf3af11c871882ea4750e60b3 Reviewed-on: http://gerrit.cloudera.org:8080/20719 Tested-by: Impala Public Jenkins <[email protected]> Reviewed-by: Kurt Deschler <[email protected]> Reviewed-by: Zihao Ye <[email protected]> Reviewed-by: Michael Smith <[email protected]> --- M be/src/benchmarks/row-batch-serialize-benchmark.cc M be/src/runtime/krpc-data-stream-sender.cc M be/src/runtime/krpc-data-stream-sender.h M be/src/runtime/outbound-row-batch.h M be/src/runtime/row-batch-serialize-test.cc M be/src/runtime/row-batch.cc M be/src/runtime/row-batch.h M fe/src/main/java/org/apache/impala/planner/DataStreamSink.java 8 files changed, 135 insertions(+), 96 deletions(-) Approvals: Impala Public Jenkins: Verified Kurt Deschler: Looks good to me, but someone else must approve Zihao Ye: Looks good to me, but someone else must approve Michael Smith: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/20719 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I64854a350a9dae8bf3af11c871882ea4750e60b3 Gerrit-Change-Number: 20719 Gerrit-PatchSet: 16 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Zihao Ye <[email protected]>
