Alex Behm has posted comments on this change. Change subject: IMPALA-2905: Handle coordinator fragment lifecycle like all others ......................................................................
Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/4402/6/be/src/exec/push-pull-sink.cc File be/src/exec/push-pull-sink.cc: Line 31: constexpr int32_t QUEUE_DEPTH = 16; > We need to carefully consider the implications on memory management. Some e Thinking about this a little more, I think the accumulation of batches without copying the tuple data in certain situations is wrong. It breaks our current memory-transfer contract. We have a similar problem in the nested-loop-join where we lazily copy the tuple data for all accumulated batches when a batch with "needs_to_return" is encountered. I think we need to rethink the design to fit our current memory transfer model. -- To view, visit http://gerrit.cloudera.org:8080/4402 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ibb0064ec2f085fa3a5598ea80894fb489a01e4df Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Henry Robinson <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Marcel Kornacker <[email protected]> Gerrit-HasComments: Yes
