bq.Consider a case when there is one normal priority request (submitted at t1) with deadline X in the queue and one more high priority request (Submitted at t2 >= t1 + x ) with zero deadline. In the current implementation the normal priority request will be offered from the queue
I would say the deadline based impl would queue up the one with 0 deadline as the prioritized one rather than the normal one. See this BoundedPriorityBlockingQueue. Am I missing something here? But if you want to implement any priority based scheduling then you can have a new scheduler and make your own queue implementation out of it. On Thu, May 28, 2015 at 3:08 PM, ashish singhi <[email protected]> wrote: > Thanks for the quick responses. > > @Anoop: > Yes, we were also thinking of implementing it along the same lines. > > @Ram: > High priority request should be always given preference over all other > scan request. > As per my understanding in the current implementation of deadline the > scenario cannot be satisfied. Consider a case when there is one normal > priority request (submitted at t1) with deadline X in the queue and one > more high priority request (Submitted at t2 >= t1 + x ) with zero deadline. > In the current implementation the normal priority request will be offered > from the queue. But our scenario require high priority request to be > offered. So there is a need of some more decision parameters (such as > priority) in the Scheduler logic. > Please correct me If I am missing something. > > Vladimir Rodionov wrote > > Scanner prioritization is not sufficient, there are a lot of other IO > activity (compaction, flushing, logging), which can severely affect high > priority scanners. > > > > HBase IO scheduler anyone? > > Yes I agree, Scan priority alone is not sufficient enough but its required > at first place. With stuffs like IO scheduler the solution will be complete. > > Regards, > Ashish Singhi > > > -----Original Message----- > From: ramkrishna vasudevan [mailto:[email protected]] > Sent: 28 May 2015 12:18 > To: [email protected] > Subject: Re: Scan priority based on type of request origin > > Atleast in trunk it uses SimpleRPCScheduler and internally it tries to use > the Priority based queue so that based on the deadline set for every > request it can schedule the highest priority RPC calls. > > I would think that your current requirement may fit with the existing one? > Except that those calls should be given higher prioirty or rather a > deadline such it is scheduled first? > > On Thu, May 28, 2015 at 12:14 PM, Anoop John <[email protected]> > wrote: > > > We have pluggable RPC scheduler now. See config > > 'hbase.region.server.rpc.scheduler.factory.class' > > > > By default a SimpleRpcScheduler is in use and there as you said, there > > are prioritization for super user. Other prioritization is based on > > the RS side method prioritization based on annotation. > > > > You can see how/whether u can have a custom RPC scheduler and queuing > > mechanism using which you can fulfill this. > > > > -Anoop- > > > > On Thu, May 28, 2015 at 11:56 AM, ashish singhi > > <[email protected]> > > wrote: > > > > > Hello folks. > > > > > > We had a customer scenario to support scan priority based on type of > > > request origin. Consider the scenario , where in n number of scan > > requests > > > are generated from different clients. Some of these requests are > > > high priority as they are originated from high privileged users OR > > > from data viewer Web UI etc. Such request should not wait and must > > > finish execution at the earliest compare to other requests. > > > > > > We went through the code and found that HBase support for > > > de-prioritizing long running scanner(HBASE-10993) and providing > > > super user rpc requests higher priority comparing to other > > > users(HBASE-13375). So this means > > apart > > > from super user all other user request will be treated as same and > > > with normal QOS. > > > > > > Let us know if there are any existing solutions already available > > > which may be we are missing. > > > > > > If implementing this make sense , we can implement this in community. > > > > > > Regards, > > > Ashish Singhi > > > > > >
