Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-3105: avoid overrunning allocated tuple buffer ......................................................................
IMPALA-3105: avoid overrunning allocated tuple buffer This change reworks allocation of tuple buffers. Previously the code tried to estimate via MaxTupleBufferSize() the maximum size of buffer it would need to allocate, assuming that AT_CAPACITY_MEM_USAGE was not exceeded. However, AT_CAPACITY_MEM_USAGE is a soft limit so can be exceeded. The fix is to change the code to set the batch's 'capacity_', which is a hard limit, to a value such that FIXED_LEN_BUFFER_LIMIT <= AT_CAPACITY_MEM_USAGE won't be exceeded. Change-Id: Idfd9cd681875821c1c379d97586d3f4850aae622 Reviewed-on: http://gerrit.cloudera.org:8080/2473 Reviewed-by: Tim Armstrong <[email protected]> Tested-by: Internal Jenkins --- M be/src/exec/data-source-scan-node.cc M be/src/exec/data-source-scan-node.h M be/src/exec/hbase-scan-node.cc M be/src/exec/hbase-scan-node.h M be/src/exec/hdfs-scanner.cc M be/src/exec/kudu-scanner.cc M be/src/exec/union-node.cc M be/src/runtime/row-batch.cc M be/src/runtime/row-batch.h 9 files changed, 77 insertions(+), 70 deletions(-) Approvals: Internal Jenkins: Verified Tim Armstrong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/2473 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Idfd9cd681875821c1c379d97586d3f4850aae622 Gerrit-PatchSet: 10 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Casey Ching <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Tim Armstrong <[email protected]>
