That is right. So if your implementation is just based on the priority of the user then the queue will be fetching the request coming out of a higher priority user always.
If there are multiple requests from the higher priority user/users - how will you distinguish between those requests? Within users of same priority you will again categorise it based on time? Regards Ram On Mon, Jun 1, 2015 at 3:11 PM, ashish singhi <[email protected]> wrote: > @Ram: > > I would say the deadline based impl would queue up the one with 0 > deadline as the prioritized one rather than the normal one. > No. The actual deadline also includes the current timestamp. So the > request which has before timestamp will be first in the queue in this case. > > > But if you want to implement any priority based scheduling then you can > have a new scheduler and make your own queue implementation out of it. > Yeah. > > Regards, > Ashish Singhi > > > -----Original Message----- > From: ramkrishna vasudevan [mailto:[email protected]] > Sent: 28 May 2015 23:43 > To: [email protected] > Subject: Re: Scan priority based on type of request origin > > bq.Consider a case when there is one normal priority request (submitted at > t1) with deadline X in the queue and one more high priority request > (Submitted at t2 >= t1 + x ) with zero deadline. In the current > implementation the normal priority request will be offered from the queue > > I would say the deadline based impl would queue up the one with 0 deadline > as the prioritized one rather than the normal one. See this > BoundedPriorityBlockingQueue. Am I missing something here? > > But if you want to implement any priority based scheduling then you can > have a new scheduler and make your own queue implementation out of it. > > On Thu, May 28, 2015 at 3:08 PM, ashish singhi <[email protected]> > wrote: > > > Thanks for the quick responses. > > > > @Anoop: > > Yes, we were also thinking of implementing it along the same lines. > > > > @Ram: > > High priority request should be always given preference over all other > > scan request. > > As per my understanding in the current implementation of deadline the > > scenario cannot be satisfied. Consider a case when there is one normal > > priority request (submitted at t1) with deadline X in the queue and > > one more high priority request (Submitted at t2 >= t1 + x ) with zero > deadline. > > In the current implementation the normal priority request will be > > offered from the queue. But our scenario require high priority request > > to be offered. So there is a need of some more decision parameters > > (such as > > priority) in the Scheduler logic. > > Please correct me If I am missing something. > > > > Vladimir Rodionov wrote > > > Scanner prioritization is not sufficient, there are a lot of other > > > IO > > activity (compaction, flushing, logging), which can severely affect > > high priority scanners. > > > > > > HBase IO scheduler anyone? > > > > Yes I agree, Scan priority alone is not sufficient enough but its > > required at first place. With stuffs like IO scheduler the solution will > be complete. > > > > Regards, > > Ashish Singhi > > > > > > -----Original Message----- > > From: ramkrishna vasudevan [mailto:[email protected]] > > Sent: 28 May 2015 12:18 > > To: [email protected] > > Subject: Re: Scan priority based on type of request origin > > > > Atleast in trunk it uses SimpleRPCScheduler and internally it tries to > > use the Priority based queue so that based on the deadline set for > > every request it can schedule the highest priority RPC calls. > > > > I would think that your current requirement may fit with the existing > one? > > Except that those calls should be given higher prioirty or rather a > > deadline such it is scheduled first? > > > > On Thu, May 28, 2015 at 12:14 PM, Anoop John <[email protected]> > > wrote: > > > > > We have pluggable RPC scheduler now. See config > > > 'hbase.region.server.rpc.scheduler.factory.class' > > > > > > By default a SimpleRpcScheduler is in use and there as you said, > > > there are prioritization for super user. Other prioritization is > > > based on the RS side method prioritization based on annotation. > > > > > > You can see how/whether u can have a custom RPC scheduler and > > > queuing mechanism using which you can fulfill this. > > > > > > -Anoop- > > > > > > On Thu, May 28, 2015 at 11:56 AM, ashish singhi > > > <[email protected]> > > > wrote: > > > > > > > Hello folks. > > > > > > > > We had a customer scenario to support scan priority based on type > > > > of request origin. Consider the scenario , where in n number of > > > > scan > > > requests > > > > are generated from different clients. Some of these requests are > > > > high priority as they are originated from high privileged users OR > > > > from data viewer Web UI etc. Such request should not wait and must > > > > finish execution at the earliest compare to other requests. > > > > > > > > We went through the code and found that HBase support for > > > > de-prioritizing long running scanner(HBASE-10993) and providing > > > > super user rpc requests higher priority comparing to other > > > > users(HBASE-13375). So this means > > > apart > > > > from super user all other user request will be treated as same and > > > > with normal QOS. > > > > > > > > Let us know if there are any existing solutions already available > > > > which may be we are missing. > > > > > > > > If implementing this make sense , we can implement this in community. > > > > > > > > Regards, > > > > Ashish Singhi > > > > > > > > > >
