Alexey Serbin has uploaded a new change for review. http://gerrit.cloudera.org:8080/4308
Change subject: [c++ client] AUTO_FLUSH_BACKGROUND optimizations ...................................................................... [c++ client] AUTO_FLUSH_BACKGROUND optimizations Some optimizations after initial performance testing of the Kudu C++ client library with AUTO_FLUSH_BACKGROUND flush mode support. The most important tuning is the default flush watermark for the mutation buffer. Chaning it from 80% to 50% gave good performance boost. The data below is for workloads of 8M rows like (int64, int32, string, string, int) where strings are about 32 bytes long in average. Each thread ran its own single-session KuduClient, where each session was running in AUTO_FLUSH_BACKGROUND flush mode. 1-thread insertion (8M rows per thread) 80% watermark: total : 35229.7 ms per row: 0.00440372 ms 50% watermark: total : 22562.8 ms per row: 0.00282035 ms 2-thread insertion (4M rows per thread) 80% watermark: total : 19683.6 ms per row: 0.00246046 ms 50% watermark: total : 12931.8 ms per row: 0.00161647 ms 4-thread insertion (2M rows per thread) 80% watermark: total : 11941.9 ms per row: 0.00149274 ms 50% watermark: total : 7724.68 ms per row: 0.000965585 ms Other related session parameters: mutation buffer size: 7M (default) maximum number of batchers: 2 (default) time-based flush interval: 1 second (default) The tests were run at ve0518.halxg.cloudera.com against binaries built in release configuration at the same machine. Change-Id: I1f0aa6d02c51bb063498709e8570e8c7214a31a0 --- M src/kudu/client/batcher.cc M src/kudu/client/batcher.h M src/kudu/client/session-internal.cc M src/kudu/client/write_op.cc 4 files changed, 9 insertions(+), 12 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/08/4308/1 -- To view, visit http://gerrit.cloudera.org:8080/4308 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1f0aa6d02c51bb063498709e8570e8c7214a31a0 Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Alexey Serbin <aser...@cloudera.com>