Hello Kudu Jenkins, Andrew Wong,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16036
to look at the new patch set (#3).
Change subject: WIP [perf] KUDU-3140 Heuristics to disable predicate evaluation
for Bloom filter
......................................................................
WIP [perf] KUDU-3140 Heuristics to disable predicate evaluation for Bloom filter
Column predicate evaluation can be expensive and ineffective column predicates
can waste CPU. TPCH Q9 exhibits significant regression of 50-96% on enabling
Bloom filter predicates. See KUDU-3140 for details.
This change adds simple heuristic taken from HDFS scanner in Impala
that basically checks for every 16 blocks and if a predicate has rejected
less than 10% of the rows scanned then disables the predicate.
The stats collection and enforcement is enabled only for disableable
predicate types, Bloom filter for now.
With Bloom filter predicate type, false positives are expected so
client is expected to do further filtering to remove false positives.
Kudu makes the decision to disable the predicate independently and doesn't
inform the client in this change which is okay for Bloom filter given
the rationale above.
TODO:
- Run TPCH with this change to determine whether this change helps Q9
that exhibits regression and doesn't regress other queries.
- Need to determine whether these heuristics don't slow down other predicates
(Only disableable predicates are tracked with latest revision so this
shouldn't be a concern)
- Update Bloom filter predicate client docs if we see improvement above.
Change-Id: I10197800a01a1b34c7821ac879caf8d272cab8dd
---
M src/kudu/common/CMakeLists.txt
M src/kudu/common/column_materialization_context.h
M src/kudu/common/generic_iterators-test.cc
M src/kudu/common/generic_iterators.cc
A src/kudu/common/predicate_effectiveness.cc
A src/kudu/common/predicate_effectiveness.h
6 files changed, 460 insertions(+), 12 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/36/16036/3
--
To view, visit http://gerrit.cloudera.org:8080/16036
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I10197800a01a1b34c7821ac879caf8d272cab8dd
Gerrit-Change-Number: 16036
Gerrit-PatchSet: 3
Gerrit-Owner: Bankim Bhavsar <[email protected]>
Gerrit-Reviewer: Andrew Wong <[email protected]>
Gerrit-Reviewer: Bankim Bhavsar <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
