Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4308
to look at the new patch set (#3).
Change subject: [c++ client] AUTO_FLUSH_BACKGROUND optimizations
......................................................................
[c++ client] AUTO_FLUSH_BACKGROUND optimizations
Optimizations after initial performance testing of the
Kudu C++ client library with AUTO_FLUSH_BACKGROUND flush mode support.
The most important tuning is the default flush watermark
for the mutation buffer. Changing it from 80% to 50% gave near 30%
performance boost in throughput for scenarios when a client pushes
data to the server as fast as it can, using workloads of 8M rows like
(int64, int32, string, string, int) where strings are about 32 bytes
long in average. Each thread ran its own single-session KuduClient,
where each session was running in AUTO_FLUSH_BACKGROUND flush mode.
1-thread insertion (8M rows per thread)
80% watermark:
total : 35229.7 ms
per row: 0.00440372 ms
50% watermark:
total : 22562.8 ms
per row: 0.00282035 ms
2-thread insertion (4M rows per thread)
80% watermark:
total : 19683.6 ms
per row: 0.00246046 ms
50% watermark:
total : 12931.8 ms
per row: 0.00161647 ms
4-thread insertion (2M rows per thread)
80% watermark:
total : 11941.9 ms
per row: 0.00149274 ms
50% watermark:
total : 7724.68 ms
per row: 0.000965585 ms
Other related session parameters:
mutation buffer size: 7M (default)
maximum number of batchers: 2 (default)
time-based flush interval: 1 second (default)
The tests were run dual Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
(12 cores per CPU) with 98GiB of memory.
This is a follow-up for 93be1310d227cf05025864654ca3f6713c2ddc2c.
Change-Id: I1f0aa6d02c51bb063498709e8570e8c7214a31a0
---
M src/kudu/client/batcher.cc
M src/kudu/client/batcher.h
M src/kudu/client/client.h
M src/kudu/client/session-internal.cc
M src/kudu/client/write_op.cc
5 files changed, 10 insertions(+), 13 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/08/4308/3
--
To view, visit http://gerrit.cloudera.org:8080/4308
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f0aa6d02c51bb063498709e8570e8c7214a31a0
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Alexey Serbin <aser...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <aser...@cloudera.com>
Gerrit-Reviewer: David Ribeiro Alves <dral...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
