Todd Lipcon has submitted this change and it was merged.

Change subject: Predicate evaluation pushdown

Predicate evaluation pushdown

The premise of this patch is to avoid the excessive use of CPU when
evaluating column predicates in specific cases. Dictionary blocks, for
instance, can evaluate predicates by checking whether a string's
codeword matches with the predicate, rather than by doing string

This patch uses a bitset to represent the set of codewords that match
a given predicate on dictionary-encoded columns. Certain decoders now
have the ability to evaluate a predicate without eagerly copying all
of their underlying data into a buffer first. Rather, the decoders can
first evaluate the predicate and copy only when needed.

Since dictionary encoding relies on plain encoding when a dictionary
gets too large, plain encoding also supports decoder-level evaluation.
While lacking the benefit from reduced string comparisons, this
optimization still improves scan speeds by avoiding excessive copies.

See the performance doc for a look into the performance differences
for dictionary encoding and plain encoding:

See the design-doc for predicate-eval-pushdown for a brief overview of the
considered implementations:

More in-depth analysis and benchmarking in upcoming blog post.

Change-Id: I31e4cce21e99f63b089d7c84410af8ed914cb576
Tested-by: Kudu Jenkins
Reviewed-by: Todd Lipcon <>
M src/kudu/cfile/
M src/kudu/cfile/binary_dict_block.h
M src/kudu/cfile/
M src/kudu/cfile/binary_plain_block.h
M src/kudu/cfile/block_encodings.h
M src/kudu/cfile/cfile-test-base.h
M src/kudu/cfile/
M src/kudu/cfile/
M src/kudu/cfile/cfile_reader.h
M src/kudu/cfile/
M src/kudu/cfile/
A src/kudu/common/column_materialization_context.h
M src/kudu/common/
M src/kudu/common/column_predicate.h
M src/kudu/common/
M src/kudu/common/
M src/kudu/common/generic_iterators.h
M src/kudu/common/iterator.h
M src/kudu/common/rowblock.h
M src/kudu/common/schema.h
M src/kudu/tablet/CMakeLists.txt
M src/kudu/tablet/
M src/kudu/tablet/
M src/kudu/tablet/cfile_set.h
M src/kudu/tablet/
M src/kudu/tablet/delta_applier.h
M src/kudu/tablet/
M src/kudu/tablet/delta_iterator_merger.h
M src/kudu/tablet/delta_store.h
M src/kudu/tablet/
M src/kudu/tablet/deltafile.h
M src/kudu/tablet/
M src/kudu/tablet/deltamemstore.h
A src/kudu/tablet/
M src/kudu/tablet/tablet-test-util.h
35 files changed, 930 insertions(+), 124 deletions(-)

  Todd Lipcon: Looks good to me, approved
  Kudu Jenkins: Verified

To view, visit
To unsubscribe, visit

Gerrit-MessageType: merged
Gerrit-Change-Id: I31e4cce21e99f63b089d7c84410af8ed914cb576
Gerrit-PatchSet: 17
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Andrew Wong <>
Gerrit-Reviewer: Alexey Serbin <>
Gerrit-Reviewer: Andrew Wong
Gerrit-Reviewer: Dan Burkert <>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <>

Reply via email to