Tim Armstrong has posted comments on this change. (
http://gerrit.cloudera.org:8080/10315 )
Change subject: IMPALA-6975: TestRuntimeRowFilters.test_row_filters failing
with Memory limit exceeded
......................................................................
Patch Set 1: Code-Review+2
I reintroduced the bug that the regression test was targeting and it fails with
the new mem_limit, so looks good to me.
diff --git a/be/src/exec/hdfs-parquet-scanner.cc
b/be/src/exec/hdfs-parquet-scanner.cc
index 8cec3d2..2661166 100644
--- a/be/src/exec/hdfs-parquet-scanner.cc
+++ b/be/src/exec/hdfs-parquet-scanner.cc
@@ -529,7 +529,7 @@ Status HdfsParquetScanner::EvaluateStatsConjuncts(
}
// Free any expr result allocations accumulated during conjunct evaluation.
- context_->expr_results_pool()->Clear();
+ // context_->expr_results_pool()->Clear();
return Status::OK();
}
@@ -863,7 +863,7 @@ Status HdfsParquetScanner::EvalDictionaryFilters(const
parquet::RowGroup& row_gr
if (dict_idx % 1024 == 0) {
// Don't let expr result allocations accumulate too much for large
dictionaries or
// many row groups.
- context_->expr_results_pool()->Clear();
+ // context_->expr_results_pool()->Clear();
}
dictionary->GetValue(dict_idx, slot);
@@ -878,7 +878,7 @@ Status HdfsParquetScanner::EvalDictionaryFilters(const
parquet::RowGroup& row_gr
}
}
// Free all expr result allocations now that we're done with the filter.
- context_->expr_results_pool()->Clear();
+ // context_->expr_results_pool()->Clear();
if (!column_has_match) {
// The column contains no value that matches the conjunct. The row group
@@ -978,7 +978,7 @@ Status HdfsParquetScanner::CommitRows(RowBatch* dst_batch,
int num_rows) {
RETURN_IF_ERROR(state_->GetQueryStatus());
// Clear expr result allocations for this thread to avoid accumulating too
much
// memory from evaluating the scanner conjuncts.
- context_->expr_results_pool()->Clear();
+ // context_->expr_results_pool()->Clear();
return Status::OK();
}
--
To view, visit http://gerrit.cloudera.org:8080/10315
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie39482e2a0aee402ce156b11cce51038cff5e61a
Gerrit-Change-Number: 10315
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil <[email protected]>
Gerrit-Reviewer: Sailesh Mukil <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
Gerrit-Comment-Date: Fri, 04 May 2018 23:11:39 +0000
Gerrit-HasComments: No
