Hello Kudu Jenkins, Andrew Wong, Bankim Bhavsar, Todd Lipcon,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16063
to look at the new patch set (#6).
Change subject: [util] improved performance of BlockingQueue
......................................................................
[util] improved performance of BlockingQueue
Updated the interface of the BlockingQueue adding methods with the move
semantics. Switched from std::list to std::deque for the underlying
queue. Other minor cleanup.
This update provides measurable performance boost for BlockingQueue
as per BlockingQueueTest.MultiThreadPerf scenario of the
blocking_queue-test. I ran the following 3 times on the same
machine (RELEASE build):
./bin/blocking_queue-test --gtest_filter='BlockingQueueTest.MultiThreadPerf'
--num_blocking_writers=4 --num_blocking_readers=4 --num_non_blocking_writers=0
Before this patch:
BlockingGet() rate: 238495 calls/sec
BlockingPut() rate: 238495 calls/sec
total Blocking{Get,Put}() rate: 476990 calls/sec
BlockingGet() rate: 223278 calls/sec
BlockingPut() rate: 223278 calls/sec
total Blocking{Get,Put}() rate: 446556 calls/sec
BlockingGet() rate: 227804 calls/sec
BlockingPut() rate: 227804 calls/sec
total Blocking{Get,Put}() rate: 455608 calls/sec
After this patch:
BlockingGet() rate: 328264 calls/sec
BlockingPut() rate: 328264 calls/sec
total Blocking{Get,Put}() rate: 656528 calls/sec
BlockingGet() rate: 312607.4 calls/sec
BlockingPut() rate: 312607.4 calls/sec
total Blocking{Get,Put}() rate: 625214.8 calls/sec
BlockingGet() rate: 292966.4 calls/sec
BlockingPut() rate: 292966.4 calls/sec
total Blocking{Get,Put}() rate: 585932.8 calls/sec
========================================================================
The case of non-symmetric writes/reads now looks more perfromant as well:
./bin/blocking_queue-test --gtest_filter='*BlockingQueueMultiThreadPerfTest*'
--num_blocking_writers=2 --num_blocking_readers=1 --num_non_blocking_writers=0
Before this patch:
BlockingGet() rate: 399823.8 calls/sec
BlockingPut() rate: 399823.8 calls/sec
total Blocking{Get,Put}() rate: 799647.6 calls/sec
After this patch:
BlockingGet() rate: 640630.6 calls/sec
BlockingPut() rate: 640630.6 calls/sec
total Blocking{Get,Put}() rate: 1281261.2 calls/sec
========================================================================
In addition, I ran more synthetic mt-log-test and the results show
slight improvement as well (at least, the performance didn't degrade):
./bin/mt-log-test --num_writer_threads=1 --num_batches_per_thread=1000000
--num_reader_threads=0 --num_ops_per_batch_avg=1 --verify_log=false
--gtest_filter='MultiThreadedLogTest.TestAppends'
Before this patch:
Time spent inserting 1000000 batches(1 threads, 1000000 per-thread): real
15.309s user 0.000s sys 0.000s
Time spent inserting 1000000 batches(1 threads, 1000000 per-thread): real
15.899s user 0.000s sys 0.000s
Time spent inserting 1000000 batches(1 threads, 1000000 per-thread): real
15.012s user 0.000s sys 0.001s
After this patch:
Time spent inserting 1000000 batches(1 threads, 1000000 per-thread): real
14.964s user 0.001s sys 0.000s
Time spent inserting 1000000 batches(1 threads, 1000000 per-thread): real
14.513s user 0.000s sys 0.000s
Time spent inserting 1000000 batches(1 threads, 1000000 per-thread): real
14.991s user 0.000s sys 0.000s
Change-Id: Ie80620e5e86cd72c29320096dcdcc712eea1b0f2
---
M src/kudu/consensus/log.cc
M src/kudu/consensus/log.h
M src/kudu/subprocess/server.cc
M src/kudu/subprocess/server.h
M src/kudu/util/blocking_queue-test.cc
M src/kudu/util/blocking_queue.h
6 files changed, 111 insertions(+), 84 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/63/16063/6
--
To view, visit http://gerrit.cloudera.org:8080/16063
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie80620e5e86cd72c29320096dcdcc712eea1b0f2
Gerrit-Change-Number: 16063
Gerrit-PatchSet: 6
Gerrit-Owner: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Andrew Wong <[email protected]>
Gerrit-Reviewer: Bankim Bhavsar <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <[email protected]>
