Hello Mike Percy, David Ribeiro Alves, Kudu Jenkins, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11395
to look at the new patch set (#4).
Change subject: KUDU-686 (part 2/2): use DeltaPreparer in DeltaFileIterator
......................................................................
KUDU-686 (part 2/2): use DeltaPreparer in DeltaFileIterator
This patch rewrites much of DeltaFileIterator to leverage DeltaPreparer for
delta preparation and service. Besides sharing code, the motivation is to
take advantage of the performance improvement inherent to DeltaPreparer:
decoding a batch of deltas just once in PrepareBatch() as opposed to on each
call to ApplyUpdates().
Seeing as DeltaPreparer was originally built for DMSIterator, here are the
various augmentations that were necessary to support DeltaFileIterator:
- REINSERT support, which meant more complicated deletion state tracking.
- FilterColumnIdsAndCollectDeltas support, cribbed from DeltaFileIterator.
- A templatized traits system to control which features were enabled. This
also meant templatizing both DeltaPreparer and DeltaFileIterator.
- Early out from the "apply all deltas for a row" loop when the timestamp
is no longer relevant. I modified DMSIterator to take advantage of this,
which should result in a performance improvement.
The improvement should be most noticeable on tables with wide schemas where
multiple columns are projected. In these situations, the column-by-column
ApplyUpdates() approach incurred a lot of unnecessary delta decoding. I
don't have a suitable microbenchmark to prove this, but I did run
diskrowset-test's TestDeltaApplicationPerformance under perf and the
resulting flame graphs showed the bulk of the iteration time as having moved
from ApplyUpdates() to PrepareBatch().
Change-Id: I87de52092262c4b42c1bd5107f9139edfc3888b5
---
M src/kudu/tablet/delta_key.h
M src/kudu/tablet/delta_store.cc
M src/kudu/tablet/delta_store.h
M src/kudu/tablet/deltafile.cc
M src/kudu/tablet/deltafile.h
M src/kudu/tablet/deltamemstore-test.cc
M src/kudu/tablet/deltamemstore.cc
M src/kudu/tablet/deltamemstore.h
M src/kudu/tablet/mvcc.cc
M src/kudu/tablet/mvcc.h
M src/kudu/tablet/tablet-test-util.h
11 files changed, 468 insertions(+), 399 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/95/11395/4
--
To view, visit http://gerrit.cloudera.org:8080/11395
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I87de52092262c4b42c1bd5107f9139edfc3888b5
Gerrit-Change-Number: 11395
Gerrit-PatchSet: 4
Gerrit-Owner: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: David Ribeiro Alves <davidral...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mpe...@apache.org>
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
