Repository: incubator-impala Updated Branches: refs/heads/master fcf32b584 -> 7d6b05a72
IMPALA-5819: DCHECK in HdfsTextScanner::Close() The DCHECK is only valid if the scanner successfully processes the entire scan range. If the scanner is cancelled or hits an error in-between buffers, boundary_column_ may be non-empty. The fix is to move the DCHECK to the end of FinishScanRange(), where it only executes if the scan range is fully processed. Testing: Ran TestCancellation in a loop locally for an hour. Change-Id: I3d63f0b0c1ac82103a60b68b244156b9fbd575e0 Reviewed-on: http://gerrit.cloudera.org:8080/7756 Reviewed-by: Alex Behm <[email protected]> Tested-by: Impala Public Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/7d6b05a7 Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/7d6b05a7 Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/7d6b05a7 Branch: refs/heads/master Commit: 7d6b05a7215d579f8df6c0685575662c6cfc09c4 Parents: fcf32b5 Author: Tim Armstrong <[email protected]> Authored: Mon Aug 21 09:03:17 2017 -0700 Committer: Impala Public Jenkins <[email protected]> Committed: Mon Aug 21 22:02:57 2017 +0000 ---------------------------------------------------------------------- be/src/exec/hdfs-text-scanner.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/7d6b05a7/be/src/exec/hdfs-text-scanner.cc ---------------------------------------------------------------------- diff --git a/be/src/exec/hdfs-text-scanner.cc b/be/src/exec/hdfs-text-scanner.cc index 761f06f..eea4e80 100644 --- a/be/src/exec/hdfs-text-scanner.cc +++ b/be/src/exec/hdfs-text-scanner.cc @@ -162,7 +162,6 @@ void HdfsTextScanner::Close(RowBatch* row_batch) { decompressor_->Close(); decompressor_.reset(); } - DCHECK(boundary_column_.IsEmpty()); boundary_pool_->FreeAll(); if (row_batch != nullptr) { row_batch->tuple_data_pool()->AcquireData(template_tuple_pool_.get(), false); @@ -331,6 +330,7 @@ Status HdfsTextScanner::FinishScanRange(RowBatch* row_batch) { DCHECK_EQ(num_tuples, 0); } + DCHECK(boundary_column_.IsEmpty()) << "Must finish processing boundary column"; scan_state_ = DONE; return Status::OK(); }
