[
https://issues.apache.org/jira/browse/HBASE-27766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773227#comment-17773227
]
Xiaolin Ha commented on HBASE-27766:
------------------------------------
|Using YCSB for performance test, defined params: -p maxexecutiontime=600 -p
clientbuffering=false -p readproportion=0.9 -p scanproportion=0.1 -p
readallfields=true -threads 400 -s, on two regionservers, and for table of 100
regions, per region size is about 2GB(uncompressed), the result is as follows,
with steal queue:
[OVERALL], RunTime(ms), 600196
[OVERALL], Throughput(ops/sec), 10980.414731187811
[TOTAL_GCS_PS_Scavenge], Count, 327
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 9728
[TOTAL_GC_TIME_%{_}PS_Scavenge], Time(%), 1.6208038707355599{_}
_[TOTAL_GCS_PS_MarkSweep], Count, 2_
_[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 243_
{_}[TOTAL_GC_TIME{_}%{_}PS_MarkSweep], Time(%), 0.04048677432038867{_}
_[TOTAL_GCs], Count, 329_
_[TOTAL_GC_TIME], Time(ms), 9971_
{_}[TOTAL_GC_TIME{_}%], Time(%), 1.6612906450559481
[READ], Operations, 5932076
[READ], AverageLatency(us), 19541.46970419799
[READ], MinLatency(us), 126
[READ], MaxLatency(us), 1187839
[READ], 95thPercentileLatency(us), 50463
[READ], 99thPercentileLatency(us), 130111
[READ], Return=OK, 5932076
[CLEANUP], Operations, 800
[CLEANUP], AverageLatency(us), 39.74125
[CLEANUP], MinLatency(us), 0
[CLEANUP], MaxLatency(us), 27807
[CLEANUP], 95thPercentileLatency(us), 13
[CLEANUP], 99thPercentileLatency(us), 32
[SCAN], Operations, 658325
[SCAN], AverageLatency(us), 187782.97140622034
[SCAN], MinLatency(us), 526
[SCAN], MaxLatency(us), 1017855
[SCAN], 95thPercentileLatency(us), 333055
[SCAN], 99thPercentileLatency(us), 404991
[SCAN], Return=OK, 658325
without steal queue:
[OVERALL], RunTime(ms), 600232
[OVERALL], Throughput(ops/sec), 10738.177904543576
[TOTAL_GCS_PS_Scavenge], Count, 316
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 10203
[TOTAL_GC_TIME_%{_}PS_Scavenge], Time(%), 1.6998427274787082{_}
_[TOTAL_GCS_PS_MarkSweep], Count, 2_
_[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 286_
{_}[TOTAL_GC_TIME{_}%{_}PS_MarkSweep], Time(%), 0.04764824267949726{_}
_[TOTAL_GCs], Count, 318_
_[TOTAL_GC_TIME], Time(ms), 10489_
{_}[TOTAL_GC_TIME{_}%], Time(%), 1.7474909701582058
[READ], Operations, 5799511
[READ], AverageLatency(us), 20715.723915516326
[READ], MinLatency(us), 115
[READ], MaxLatency(us), 793087
[READ], 95thPercentileLatency(us), 54847
[READ], 99thPercentileLatency(us), 130047
[READ], Return=OK, 5799511
[CLEANUP], Operations, 800
[CLEANUP], AverageLatency(us), 43.0025
[CLEANUP], MinLatency(us), 0
[CLEANUP], MaxLatency(us), 29631
[CLEANUP], 95thPercentileLatency(us), 16
[CLEANUP], 99thPercentileLatency(us), 29
[SCAN], Operations, 645887
[SCAN], AverageLatency(us), 185176.07673942964
[SCAN], MinLatency(us), 534
[SCAN], MaxLatency(us), 923135
[SCAN], 95thPercentileLatency(us), 348927
[SCAN], 99thPercentileLatency(us), 432383
[SCAN], Return=OK, 645887|
> Support steal job queue mode for read RPC queues of RWQueueRpcExecutor
> ----------------------------------------------------------------------
>
> Key: HBASE-27766
> URL: https://issues.apache.org/jira/browse/HBASE-27766
> Project: HBase
> Issue Type: Improvement
> Components: rpc
> Affects Versions: 3.0.0-alpha-3, 2.5.3
> Reporter: Xiaolin Ha
> Assignee: Xiaolin Ha
> Priority: Major
>
> Currently, the RPC queues are distinguished by request type, under most
> circumstances of RWQueueRpcExecutor, there are write queues and read queues,
> while reads queues are always divided by get requests and scan requests. The
> reason why we isolate the scan requests and get requests is that we do not
> want large scans block small gets.
> Since the handler resources for a regionserver is limited and we can't
> dynamicly change the handler ratio by the ratio of requests. We should both
> keep large scan and the small gets be isolated, and let the idle handlers for
> the samller ratio scans to handle some gets when the gets handlers are busy.
> This steal queue idea can also be used in other circumstances, e.g. idle read
> handler steal jobs from write queus.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
