[
https://issues.apache.org/jira/browse/KUDU-3782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabriella Lotz updated KUDU-3782:
---------------------------------
Description:
h3. Summary
When a tablet server serves a counting scan (COUNT/(/*/) or COUNT(<constant>),
which uses an empty projection) that also carries column predicates,
MaterializingIterator::Init (in src/kudu/common/generic_iterators.cc) computes
the number of projected columns minus the number of predicates using mismatched
signed/unsigned types. When the projected column count is smaller than the
predicate count, the subtraction underflows to a very large value, which is
passed to std::vector::reserve, throwing an uncaught std::length_error and
aborting the process.
This happens because the projection for a counting scan is tiny, while during
initialization the rowset iterator (CFileSet::Iterator::OptimizePKPredicates →
LiftPrimaryKeyBounds) re-adds predicates derived from the rowset's primary key
bounds on key columns that are not in the projection. The guarding DCHECK_GE is
compiled out in release builds, so release builds reach the underflow. It is
intermittent because it depends on the rowset's on-disk key layout.
Trigger: a COUNT(*)/COUNT(<const>) query with predicates on a non-contiguous
subset of a composite primary key, against a rowset whose primary key bounds
share a long constant prefix.
h3. Fix (in MaterializingIterator::Init)
Remove the invalid DCHECK_GE, compute the reservation so it cannot underflow,
and skip predicates whose column is not in the projection (these are
rowset-derived and already enforced by the key range) instead of failing the
scan. A regression test covering a projection smaller than the predicate set
passes with this change, and the existing generic_iterators-test suite
continues to pass.
h3. Workaround
Rewrite affected counting queries so the scan projects real columns, for
example by wrapping the count in a sub-query that selects actual rows. A
non-empty projection avoids the code path entirely. Only COUNT(...) queries
that include a WHERE clause can trigger this.
was:
h3. Summary
When a tablet server serves a counting scan (COUNT(*) or COUNT(<constant>),
which uses an empty projection) that also carries column predicates,
MaterializingIterator::Init (in src/kudu/common/generic_iterators.cc) computes
the number of projected columns minus the number of predicates using mismatched
signed/unsigned types. When the projected column count is smaller than the
predicate count, the subtraction underflows to a very large value, which is
passed to std::vector::reserve, throwing an uncaught std::length_error and
aborting the process.
This happens because the projection for a counting scan is tiny, while during
initialization the rowset iterator (CFileSet::Iterator::OptimizePKPredicates →
LiftPrimaryKeyBounds) re-adds predicates derived from the rowset's primary key
bounds on key columns that are not in the projection. The guarding DCHECK_GE is
compiled out in release builds, so release builds reach the underflow. It is
intermittent because it depends on the rowset's on-disk key layout.
Trigger: a COUNT(*)/COUNT(<const>) query with predicates on a non-contiguous
subset of a composite primary key, against a rowset whose primary key bounds
share a long constant prefix.
h3. Fix (in MaterializingIterator::Init)
Remove the invalid DCHECK_GE, compute the reservation so it cannot underflow,
and skip predicates whose column is not in the projection (these are
rowset-derived and already enforced by the key range) instead of failing the
scan. A regression test covering a projection smaller than the predicate set
passes with this change, and the existing generic_iterators-test suite
continues to pass.
h3. Workaround
Rewrite affected counting queries so the scan projects real columns, for
example by wrapping the count in a sub-query that selects actual rows. A
non-empty projection avoids the code path entirely. Only COUNT(...) queries
that include a WHERE clause can trigger this.
> TServer aborts (SIGABRT / std::length_error) on COUNT(*) scans with predicates
> ------------------------------------------------------------------------------
>
> Key: KUDU-3782
> URL: https://issues.apache.org/jira/browse/KUDU-3782
> Project: Kudu
> Issue Type: Bug
> Components: tserver
> Reporter: Gabriella Lotz
> Priority: Critical
>
> h3. Summary
> When a tablet server serves a counting scan (COUNT/(/*/) or
> COUNT(<constant>), which uses an empty projection) that also carries column
> predicates, MaterializingIterator::Init (in
> src/kudu/common/generic_iterators.cc) computes the number of projected
> columns minus the number of predicates using mismatched signed/unsigned
> types. When the projected column count is smaller than the predicate count,
> the subtraction underflows to a very large value, which is passed to
> std::vector::reserve, throwing an uncaught std::length_error and aborting the
> process.
> This happens because the projection for a counting scan is tiny, while during
> initialization the rowset iterator (CFileSet::Iterator::OptimizePKPredicates
> → LiftPrimaryKeyBounds) re-adds predicates derived from the rowset's primary
> key bounds on key columns that are not in the projection. The guarding
> DCHECK_GE is compiled out in release builds, so release builds reach the
> underflow. It is intermittent because it depends on the rowset's on-disk key
> layout.
> Trigger: a COUNT(*)/COUNT(<const>) query with predicates on a non-contiguous
> subset of a composite primary key, against a rowset whose primary key bounds
> share a long constant prefix.
> h3. Fix (in MaterializingIterator::Init)
> Remove the invalid DCHECK_GE, compute the reservation so it cannot underflow,
> and skip predicates whose column is not in the projection (these are
> rowset-derived and already enforced by the key range) instead of failing the
> scan. A regression test covering a projection smaller than the predicate set
> passes with this change, and the existing generic_iterators-test suite
> continues to pass.
> h3. Workaround
> Rewrite affected counting queries so the scan projects real columns, for
> example by wrapping the count in a sub-query that selects actual rows. A
> non-empty projection avoids the code path entirely. Only COUNT(...) queries
> that include a WHERE clause can trigger this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
