Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-4023: don't attach buffered tuple streams to batches ......................................................................
IMPALA-4023: don't attach buffered tuple streams to batches This simplifies the memory transfer model by eliminating one category of resources that can be attached. This patch also separates the concepts of attaching resources and flushing resources. Previously RowBatch::AddTupleStream() implicitly flushed resources from the ExecNode pipeline, which various ExecNodes relied on to free up memory reservations for subsequent processing. In a subsequent patch I want the FlushResources() API to become stronger: it will force streaming ExecNodes to flush their batches or forces blocking ExecNodes to acquire ownership of the memory resources. We can't do this right now since we don't have a way to transfer ownership of BufferedBlockMgr Blocks. Change-Id: I6471422f86ce71e4c6ab277a276000051bc2e8ff Reviewed-on: http://gerrit.cloudera.org:8080/4448 Reviewed-by: Tim Armstrong <tarmstr...@cloudera.com> Tested-by: Internal Jenkins --- M be/src/exec/analytic-eval-node.cc M be/src/exec/analytic-eval-node.h M be/src/exec/nested-loop-join-builder.cc M be/src/exec/nested-loop-join-node.cc M be/src/exec/partitioned-aggregation-node.cc M be/src/exec/partitioned-hash-join-builder.cc M be/src/exec/partitioned-hash-join-node.cc M be/src/runtime/buffered-tuple-stream-test.cc M be/src/runtime/buffered-tuple-stream.cc M be/src/runtime/buffered-tuple-stream.h M be/src/runtime/row-batch-test.cc M be/src/runtime/row-batch.cc M be/src/runtime/row-batch.h M be/src/runtime/sorted-run-merger.cc M be/src/runtime/sorted-run-merger.h M be/src/runtime/sorter.cc 16 files changed, 240 insertions(+), 159 deletions(-) Approvals: Internal Jenkins: Verified Tim Armstrong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/4448 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I6471422f86ce71e4c6ab277a276000051bc2e8ff Gerrit-PatchSet: 14 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
