Hello David Ribeiro Alves, Andrew Wong,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/6430
to review the following change.
Change subject: compaction: fix rowblock size in compaction output
......................................................................
compaction: fix rowblock size in compaction output
In compaction, we let the compaction input return fewer rows at a time
than the maximum (100). In this case, we resize the rowblock down to
match the number of returned rows before feeding it into the DiskRowSet
write path.
However, we weren't resetting it back _up_ to the maximum capacity
before fetching the next batch of rows. This meant that after any
"small" batch from the underlying rowsets, all future batches would be
limited to that same size.
In practice this was causing most calls to the write path to use very
small batches, causing a lot of extra function calls, hurting CPU cache
locality, etc.
This improved an end-to-end bulk-write workload about 12% (see [1] under
the "After fixing compaction bug" section). I also measured the
improvement using the compaction-test benchmark:
before:
Time spent compacting without overlap: real 0.837s user 0.505s sys 0.084s
Time spent compacting without overlap: real 0.850s user 0.507s sys 0.080s
Time spent compacting without overlap: real 0.864s user 0.493s sys 0.086s
Time spent compacting without overlap: real 0.946s user 0.491s sys 0.069s
Time spent compacting without overlap: real 1.005s user 0.489s sys 0.086s
Time spent compacting with overlap: real 1.026s user 0.678s sys 0.107s
Time spent compacting with overlap: real 1.091s user 0.683s sys 0.083s
Time spent compacting with overlap: real 1.117s user 0.691s sys 0.085s
Time spent compacting with overlap: real 1.133s user 0.705s sys 0.078s
Time spent compacting with overlap: real 1.220s user 0.695s sys 0.097s
after:
Time spent compacting without overlap: real 0.790s user 0.492s sys 0.067s
Time spent compacting without overlap: real 0.816s user 0.484s sys 0.101s
Time spent compacting without overlap: real 0.831s user 0.498s sys 0.081s
Time spent compacting without overlap: real 0.833s user 0.497s sys 0.079s
Time spent compacting without overlap: real 0.846s user 0.507s sys 0.083s
Time spent compacting with overlap: real 0.835s user 0.548s sys 0.080s
Time spent compacting with overlap: real 0.839s user 0.552s sys 0.077s
Time spent compacting with overlap: real 0.861s user 0.556s sys 0.080s
Time spent compacting with overlap: real 0.871s user 0.533s sys 0.079s
Time spent compacting with overlap: real 0.891s user 0.546s sys 0.083s
The "with overlap" case is 20-30% improved in terms of user-CPU.
I also ran perf-stat on the 'WithOverlap' case:
Before:
Performance counter stats for './build/latest/bin/compaction-test
--gtest_filter=TestCompaction.BenchmarkMergeWithOverlap*' (10 runs):
2556.524929 task-clock # 0.904 CPUs utilized
( +- 0.52% )
241 context-switches # 0.094 K/sec
( +- 1.95% )
29 cpu-migrations # 0.011 K/sec
( +- 12.97% )
55,860 page-faults # 0.022 M/sec
( +- 0.19% )
8,317,627,473 cycles # 3.253 GHz
( +- 0.51% )
<not supported> stalled-cycles-frontend
<not supported> stalled-cycles-backend
16,685,733,672 instructions # 2.01 insns per cycle
( +- 0.01% )
2,973,888,033 branches # 1163.254 M/sec
( +- 0.01% )
12,123,206 branch-misses # 0.41% of all branches
( +- 0.15% )
2.827881785 seconds time elapsed
( +- 1.04% )
After:
2397.755099 task-clock # 0.909 CPUs utilized
( +- 0.29% )
250 context-switches # 0.104 K/sec
( +- 3.79% )
23 cpu-migrations # 0.010 K/sec
( +- 11.43% )
56,062 page-faults # 0.023 M/sec
( +- 0.11% )
7,793,443,483 cycles # 3.250 GHz
( +- 0.29% )
<not supported> stalled-cycles-frontend
<not supported> stalled-cycles-backend
15,852,415,910 instructions # 2.03 insns per cycle
( +- 0.01% )
2,836,337,368 branches # 1182.914 M/sec
( +- 0.01% )
10,362,726 branch-misses # 0.37% of all branches
( +- 0.17% )
2.638284381 seconds time elapsed
( +- 1.03% )
The number of branch-misses is substantially reduced.
[1]
https://docs.google.com/document/d/1U1IXS1XD2erZyq8_qG81A1gZaCeHcq2i0unea_eEf5c/edit#heading=h.b98m0h4dgd3w
Change-Id: I6fc4002fb84a33e0032577055fa2cdc947ce4191
---
M src/kudu/tablet/compaction.cc
1 file changed, 1 insertion(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/30/6430/1
--
To view, visit http://gerrit.cloudera.org:8080/6430
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I6fc4002fb84a33e0032577055fa2cdc947ce4191
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <[email protected]>
Gerrit-Reviewer: Andrew Wong <[email protected]>
Gerrit-Reviewer: David Ribeiro Alves <[email protected]>
