[
https://issues.apache.org/jira/browse/KUDU-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15250076#comment-15250076
]
Binglin Chang commented on KUDU-1235:
-------------------------------------
I updated the code, adding block size control, netaddr hash, and option for
client per get thread.
Some result:
{noformat}
result on i7-4770 CPU @ 3.40GHz (4 core 8 thread):
netaddr hash & 1 client
-----------------------
get 16 20
I0420 15:58:54.828622 4355 get_perf-itest.cc:157] Get total: 151870 QPS:
75932.8
I0420 15:58:56.828722 4355 get_perf-itest.cc:157] Get total: 298395 QPS:
73258.8
I0420 15:58:58.828829 4355 get_perf-itest.cc:157] Get total: 446941 QPS:
74269.1
scan 16 20
I0420 15:59:15.026579 4355 get_perf-itest.cc:157] Scan total: 72227 QPS:
36112.5
I0420 15:59:17.026675 4355 get_perf-itest.cc:157] Scan total: 142824 QPS:
35296.8
I0420 15:59:19.026772 4355 get_perf-itest.cc:157] Scan total: 213818 QPS:
35495.3
netaddr hash & client per thread
--------------------------------
get 16 20
I0420 16:03:42.521334 4640 get_perf-itest.cc:157] Get total: 112199 QPS:
56097.9
I0420 16:03:44.521497 4640 get_perf-itest.cc:157] Get total: 223665 QPS:
55728.5
I0420 16:03:46.521620 4640 get_perf-itest.cc:157] Get total: 334350 QPS:
55339.1
netaddr hash & 1 client & 4096 block size
-----------------------------------------
get 16 20
I0420 16:10:55.134418 5765 get_perf-itest.cc:161] Get total: 169097 QPS:
84546.2
I0420 16:10:57.134510 5765 get_perf-itest.cc:161] Get total: 329674 QPS:
80284.8
I0420 16:10:59.134618 5765 get_perf-itest.cc:161] Get total: 490274 QPS:
80295.7
scan 16 20
I0420 16:11:47.290632 5765 get_perf-itest.cc:161] Scan total: 88014 QPS:
44005.7
I0420 16:11:49.290726 5765 get_perf-itest.cc:161] Scan total: 176982 QPS:
44481.9
I0420 16:11:51.290820 5765 get_perf-itest.cc:161] Scan total: 262770 QPS: 42892
{noformat}
# netaddr hash helps
# set smaller block size helps
# Looks like in machine with less cores, set client per thread actually make
things worse.
> Add Get API
> -----------
>
> Key: KUDU-1235
> URL: https://issues.apache.org/jira/browse/KUDU-1235
> Project: Kudu
> Issue Type: New Feature
> Reporter: Binglin Chang
> Assignee: Binglin Chang
> Attachments: perf-get.svg, perf-scan-opt.svg, perf-scan.svg
>
>
> Get API is more user friendly and efficient if use just want primary key
> lookup.
> I setup a cluster and test get/scan single row using ycsb, initial test shows
> better performance for get.
> {noformat}
> kudu_workload:
> recordcount=1000000
> operationcount=1000000
> workload=com.yahoo.ycsb.workloads.CoreWorkload
> readallfields=false
> readproportion=1
> updateproportion=0
> scanproportion=0
> insertproportion=0
> requestdistribution=uniform
> use_get_api=false
> load:
> ./bin/ycsb load kudu -P workloads/kudu_workload -p sync_ops=false -p
> pre_split_num_tablets=1 -p table_name=ycsb_wiki_example -p
> masterQuorum='c3-kudu-tst-st01.bj:32600' -threads 100
> read test:
> ./bin/ycsb run kudu -P workloads/kudu_workload -p
> masterQuorum='c3-kudu-tst-st01.bj:32600' -threads 100
> {noformat}
> Get API:
> [OVERALL], RunTime(ms), 21304.0
> [OVERALL], Throughput(ops/sec), 46939.54187007135
> [CLEANUP], Operations, 100.0
> [CLEANUP], AverageLatency(us), 423.57
> [CLEANUP], MinLatency(us), 24.0
> [CLEANUP], MaxLatency(us), 19327.0
> [CLEANUP], 95thPercentileLatency(us), 52.0
> [CLEANUP], 99thPercentileLatency(us), 18815.0
> [READ], Operations, 1000000.0
> [READ], AverageLatency(us), 2065.185152
> [READ], MinLatency(us), 134.0
> [READ], MaxLatency(us), 92159.0
> [READ], 95thPercentileLatency(us), 2391.0
> [READ], 99thPercentileLatency(us), 6359.0
> [READ], Return=0, 1000000
> Scan API:
> [OVERALL], RunTime(ms), 38259.0
> [OVERALL], Throughput(ops/sec), 26137.6408165399
> [CLEANUP], Operations, 100.0
> [CLEANUP], AverageLatency(us), 47.32
> [CLEANUP], MinLatency(us), 16.0
> [CLEANUP], MaxLatency(us), 1837.0
> [CLEANUP], 95thPercentileLatency(us), 41.0
> [CLEANUP], 99thPercentileLatency(us), 158.0
> [READ], Operations, 1000000.0
> [READ], AverageLatency(us), 3595.825249
> [READ], MinLatency(us), 139.0
> [READ], MaxLatency(us), 3139583.0
> [READ], 95thPercentileLatency(us), 3775.0
> [READ], 99thPercentileLatency(us), 7659.0
> [READ], Return=0, 1000000
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
