[
https://issues.apache.org/jira/browse/IMPALA-6187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Ho resolved IMPALA-6187.
--------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.11.0
https://github.com/apache/impala/commit/63f17e9ceaed92a28ea12567a36b746e54fffdb3
IMPALA-6187: Fix missing conjuncts evaluation with empty projection
Previously, scanners will assume that there are no conjuncts associated
with a scan node for queries with no materialized slots (e.g. count(*)).
This is not necessarily the case as one can write queries such as
select count(*) from tpch.lineitem where rand() * 10 < 0; or
select count(*) from tpch.lineitem where rand() > <a partition column>.
In which case, the conjuncts should still be evaluated once per row.
This change fixes the problem in the short-circuit handling logic for
count(*) to evaluate the conjuncts once per row and only commits a row
to the output row batch if the conjuncts evaluate to true.
Testing done: Added the example above to the scanner test
Change-Id: Ib530f1fdcd2c6de699977db163b3f6eb38481517
Reviewed-on: http://gerrit.cloudera.org:8080/8623
Reviewed-by: Tim Armstrong <[email protected]>
Reviewed-by: Alex Behm <[email protected]>
Reviewed-by: Dan Hecht <[email protected]>
Tested-by: Impala Public Jenkins
> Scan with conjuncts but no materialized slots crashes Impalad
> -------------------------------------------------------------
>
> Key: IMPALA-6187
> URL: https://issues.apache.org/jira/browse/IMPALA-6187
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.10.0
> Reporter: Michael Ho
> Assignee: Michael Ho
> Priority: Critical
> Labels: correctness
> Fix For: Impala 2.11.0
>
>
> The simple query {noformat} select count(*) from tpch_parquet.lineitem where
> rand() = 10.0 {noformat} will trigger the following DCHECK below in
> {{HdfsScanner::WriteTemplate()}}:
> {noformat}
> int HdfsScanner::WriteTemplateTuples(TupleRow* row, int num_tuples) {
> DCHECK_GE(num_tuples, 0);
> DCHECK_EQ(scan_node_->tuple_idx(), 0);
> DCHECK_EQ(conjunct_evals_->size(), 0); <<------
> if (num_tuples == 0 || template_tuple_ == NULL) return num_tuples;
> Tuple** row_tuple = reinterpret_cast<Tuple**>(row);
> for (int i = 0; i < num_tuples; ++i) row_tuple[i] = template_tuple_;
> return num_tuples;
> }
> {noformat}
> The assumption in the scanner is that if there are no materialized slots
> (e.g. count star), there should be no conjuncts too but that doesn't always
> seems to be the case.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
