Hello Tim Armstrong,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/2203
to look at the new patch set (#15).
Change subject: IMPALA-2399: Check for mem limit in allocations in parquet
scanner and decompressor
......................................................................
IMPALA-2399: Check for mem limit in allocations in parquet scanner and
decompressor
This change replaces all calls to MemPool::Allocate() with
MemPool::TryAllocate() in the parquet scanner and the decompressor.
Also streamline CheckQueryState() to avoid unnecessary spinlock
acquisition for the common case when there is no error. Also
removes some dead code in the text converter.
MemPool::Allocate() is also updated to return a valid pointer
instead of NULL when the allocation size is zero. NULL is only
returned during allocation failure.
This change also updates CollectionValueBuilder::GetFreeMemory()
to return Status in case it exceeds memory limit. As part of the
change, the max allocation limit (2 GB) is also removed from it
as 64-bit allocations are supported in MemPool with this change.
Change-Id: Ic70400407b7662999332448f4d1bce2cc344ca89
---
M be/src/exec/base-sequence-scanner.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-scan-node.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/hdfs-scanner.h
M be/src/exec/hdfs-text-scanner.cc
M be/src/exec/text-converter.cc
M be/src/exec/text-converter.h
M be/src/runtime/buffered-tuple-stream-test.cc
M be/src/runtime/collection-value-builder-test.cc
M be/src/runtime/collection-value-builder.h
M be/src/runtime/collection-value.h
M be/src/runtime/mem-pool.cc
M be/src/runtime/mem-pool.h
M be/src/runtime/mem-tracker.cc
M be/src/runtime/mem-tracker.h
M be/src/runtime/row-batch-serialize-test.cc
M be/src/runtime/runtime-state.cc
M be/src/runtime/runtime-state.h
M be/src/util/codec.h
M be/src/util/decompress.cc
M be/src/util/runtime-profile.h
M common/thrift/generate_error_codes.py
23 files changed, 335 insertions(+), 194 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/03/2203/15
--
To view, visit http://gerrit.cloudera.org:8080/2203
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic70400407b7662999332448f4d1bce2cc344ca89
Gerrit-PatchSet: 15
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Michael Ho <[email protected]>
Gerrit-Reviewer: Dan Hecht <[email protected]>
Gerrit-Reviewer: Michael Ho <[email protected]>
Gerrit-Reviewer: Skye Wanderman-Milne <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
