Ah ok. So if I understand you right, irrespective of which queue type I use, if a get comes when a chunk is being processed, it is going to wait.
> what fifo vs deadline does is pushing the scan chunk back in the queue if you have multiple operation with higher priority. So, if I set the queue type to FIFO, the scans shouldn't get pushed back correct? Wouldn't this mean that a "get" would be after a scan in the queue if the scan was submitted before? Or is it still possible in this case for the handler to switch between the scan chunks, quickly execute the "get" and then get back to execute the rest of the chunks? Also following Ram's suggestion, another thing I found a little confusing from the UI is that I currently have the RPC handlers to set to 1 on 1 RS. Still for some reason my UI is showing multiple handlers in a waiting state even after restarting. On Fri, Sep 4, 2015 at 11:18 AM, Matteo Bertozzi <[email protected]> wrote: > scan is chunked in small pieces, it is not a single long operation. > what fifo vs deadline does is pushing the scan chunk back in the queue > if you have multiple operation with higher priority. > > after N runs of scanner.next() you'll be pushed back following a > sqrt(numOfNext * weight) > http://blog.cloudera.com/wp-content/uploads/2014/11/hbase-multi-f2.png > > so, the more next you call the further your scan chunk will be delayed. > but once your scan chunk is in the executor, there is no way (at the > moment) > to eject that task, so if you have a single thread and you have an > hi-priority get > that get has to wait the scan chunk to be executed. > > unless you have tons of requests to fill the queue > and a super-long scan (or you increase the weight to simulate that) > you'll probably not be able to notice any difference. > > On Fri, Sep 4, 2015 at 9:05 AM, ramkrishna vasudevan < > [email protected]> wrote: > > > Hi Kulkarni > > > > Which version of HBase are you using? Just to confirm are you running as > > super user only? I think there are some meta handlers also with which > > super user calls are handled. > > > > From the UI are you able to monitor the handlers and see how many are > > active and what is the status of it. > > > > Regards > > Ram > > > > On Fri, Sep 4, 2015 at 7:42 PM, [email protected] < > > [email protected]> wrote: > > > > > Hello all, > > > > > > So I am trying to play around with de-prioritizing long running > scanners > > > feature added with HBASE-10993. Essentially I wanted to see how > > "deadline" > > > queue prevents contention. This is what my did for my setup. Changed > the > > > callqueue type to "fifo" and reduced the number of RPC handlers to 1 to > > > ensure that I have only one RPC queue in place. I am also testing on a > > > cluster with only one RS to again make sure that there is only one RPC > > > handler working. I confirmed that my settings are properly in place > with > > > the following message in the hmaster logs after I restarted my cluster. > > > > > > "INFO org.apache.hadoop.hbase.ipc.SimpleRpcScheduler: Using fifo as > user > > > call queue, count=1" > > > > > > I now kicked off a long running "scan" first and while the scan was > > running > > > kicked off another "get". My hope was that with fifo, we should see a > > > contention and the "get" would essentially hang until the "scan" > finished > > > as it would be placed behind the "scan" request in the RPC queue. But I > > see > > > the gets still return quickly irrespective of whether the scan is > running > > > or not. > > > > > > Is there something obvious that I am missing here to be able to > reproduce > > > the contention? > > > > > > Thanks, > > > Swarnim > > > > > > -- Swarnim
