This is an automated email from the ASF dual-hosted git repository.
alexey pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
The following commit(s) were added to refs/heads/master by this push:
new e77733e table_locations-itest: reuse protobufs in the benchmark client
e77733e is described below
commit e77733ef37e981a9e1b1d2787868ed08b87e2603
Author: Todd Lipcon <[email protected]>
AuthorDate: Fri Jul 24 10:52:05 2020 -0700
table_locations-itest: reuse protobufs in the benchmark client
The previous client code would allocate a new request/response protobuf
for each call, which, while maybe realistic from a client perspective,
ended up causing a lot of allocator contention which also slowed down
the server's ability to allocate, making for a less realistic
benchmark/stress test of the server.
This patch simply reuses the protobufs to reduce allocator pressure,
speeding up the benchmark significantly. Benchmarked with:
$ KUDU_ALLOW_SLOW_TESTS=1 ./build/latest/bin/table_locations-itest \
--gtest_filter=TableLocationsTest.GetTableLocationsBenchmark \
--rpc_num_service_threads=32 \
--benchmark_num_threads=48
RPC benchmark:
==================
before:
RPC queue overflows: 0
GetTableLocations RPC: 43240.8 req/sec
Stats on GetTableLocations RPC (times in microseconds): Count: 216204
Mean: 308.099
Percentiles:
0% (min) = 15
25% = 146
50% (med) = 243
75% = 392
95% = 744
99% = 1120
99.9% = 2080
99.99% = 4576
100% (max) = 42197
after:
GetTableLocations RPC: 57670 req/sec
Stats on GetTableLocations RPC (times in microseconds): Count: 288350
Mean: 248.583
Percentiles:
0% (min) = 19
25% = 152
50% (med) = 208
75% = 296
95% = 520
99% = 788
99.9% = 1736
99.99% = 4320
100% (max) = 8313
Function benchmark:
=====================
Before:
GetTableLocations function call: 264899.2 req/sec
After:
GetTableLocations function call: 327872.2 req/sec
Change-Id: Iab16d3a28d295184f3e23fca1fb83d9acfe5ad19
Reviewed-on: http://gerrit.cloudera.org:8080/16236
Reviewed-by: Alexey Serbin <[email protected]>
Tested-by: Kudu Jenkins
---
src/kudu/integration-tests/table_locations-itest.cc | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/src/kudu/integration-tests/table_locations-itest.cc
b/src/kudu/integration-tests/table_locations-itest.cc
index 2014ca7..22473af 100644
--- a/src/kudu/integration-tests/table_locations-itest.cc
+++ b/src/kudu/integration-tests/table_locations-itest.cc
@@ -474,10 +474,12 @@ TEST_F(TableLocationsTest, GetTableLocationsBenchmark) {
vector<uint64_t> err_counters(kNumThreads, 0);
for (auto i = 0; i < kNumThreads; ++i) {
threads.emplace_back([&, i]() {
+ GetTableLocationsRequestPB req;
+ GetTableLocationsResponsePB resp;
while (!stop) {
- GetTableLocationsRequestPB req;
- GetTableLocationsResponsePB resp;
RpcController controller;
+ req.Clear();
+ resp.Clear();
req.mutable_table()->set_table_name(table_name);
req.set_max_returned_locations(1000);
req.set_intern_ts_infos_in_response(true);
@@ -543,9 +545,11 @@ TEST_F(TableLocationsTest,
GetTableLocationsBenchmarkFunctionCall) {
vector<uint64_t> err_counters(kNumThreads, 0);
for (size_t idx = 0; idx < kNumThreads; ++idx) {
threads.emplace_back([&, idx]() {
+ GetTableLocationsRequestPB req;
+ GetTableLocationsResponsePB resp;
while (!stop) {
- GetTableLocationsRequestPB req;
- GetTableLocationsResponsePB resp;
+ req.Clear();
+ resp.Clear();
req.mutable_table()->set_table_name(table_name);
req.set_max_returned_locations(1000);
req.set_intern_ts_infos_in_response(true);
