Michael Ho has posted comments on this change. Change subject: IMPALA-2399: Check for mem limit in allocations in parquet scanner and decompressor ......................................................................
Patch Set 14: (3 comments) http://gerrit.cloudera.org:8080/#/c/2203/14/be/src/exec/base-sequence-scanner.cc File be/src/exec/base-sequence-scanner.cc: Line 95: DCHECK(prepared_ == (batch_ != NULL)); > That's true. We can only rely on if(batch_ != NULL) and DCHECK(batch_ != NU Actually, prepared_ becomes kind of useless now so it's removed. Instead, when batch_ == NULL, DCHECK no resources are leaked (e.g. mempools and completed_io_buffers_ are empty). http://gerrit.cloudera.org:8080/#/c/2203/14/be/src/exec/hdfs-parquet-scanner.cc File be/src/exec/hdfs-parquet-scanner.cc: Line 1648: DCHECK_EQ(num_rows, 0); > Why is this DCHECK interesting? i.e. why does the caller care about num_row DCHECK removed. http://gerrit.cloudera.org:8080/#/c/2203/14/be/src/runtime/collection-value.h File be/src/runtime/collection-value.h: Line 32: int64_t num_tuples; > I think the frontend needs to know about this change. Maybe revert this cha Thanks for pointing that out. I reverted this line of change and some others in CollectionValueBuilder but updated ByteSIze() to cast num_tuples to int64_t (which I believe won't break the interface with frontend). Also, keep the removal of the size restriction of allocations in CollectionValueBuilder. -- To view, visit http://gerrit.cloudera.org:8080/2203 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ic70400407b7662999332448f4d1bce2cc344ca89 Gerrit-PatchSet: 14 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Michael Ho <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-Reviewer: Skye Wanderman-Milne <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-HasComments: Yes
