We should set the factor to be a very small number, e.g. the handler count of the regionserver is set to 200, the factor can be 0.01, while the call queue compute algorithm should be improved to make each request type have at least one call queue.
张铎(Duo Zhang) <palomino...@gmail.com> 于2023年5月30日周二 10:44写道： > Set the factor to 1 will lead to a single queue for a request type? > > Xiaolin Ha <summer.he...@gmail.com> 于2023年5月30日周二 09:24写道： > > > Currently, the handlers for processing RPC calls are divided into three > > types, they are write, read and scan. Correspondingly, each type of > handler > > corresponds to a certain number of queues. > > > > The corresponding mode of handler and queue is according to > > "hbase.ipc.server.callqueue.handler.factor", the default is 0.1, that is, > > the default ratio of the number of queues to the number of handlers is > > 1:10. The handlers for a call type are divided into handler*0.1 groups, > > while each group handlers consume the same queue. > > > > I tested the performance between single queue and multi queue mode, the > > result is that, for requests of similar size, multi queue and single > queue > > do not impact the performance too much, as displayed in > > https://issues.apache.org/jira/browse/HBASE-27683. > > But for scenarios where the request size or the process time of calls > > varies greatly, a single queue has a very big advantage, it can make full > > use of all handlers to handle all requests of the same type, such as all > > the get handlers to process all the get requests. For the call queues, > when > > the producing speed is larger than or equal to the speed of consuming, > just > > several large calls can block the processing of small calls in the same > > queue. > > We found a scenario for feature stores in our production clusters, while > > the requests/responses size of gets varies too much, single queue mode > can > > help to greatly reduce the queue time of calls. > > > > Thanks. > > >
