Kulkarni: Image didn't go through. Please use third party site for the image.
Cheers On Fri, Sep 4, 2015 at 10:03 AM, [email protected] < [email protected]> wrote: > Ah ok. So if I understand you right, irrespective of which queue type I > use, if a get comes when a chunk is being processed, it is going to wait. > > > what fifo vs deadline does is pushing the scan chunk back in the queue if > you have multiple operation with higher priority. > > So, if I set the queue type to FIFO, the scans shouldn't get pushed back > correct? Wouldn't this mean that a "get" would be after a scan in the queue > if the scan was submitted before? Or is it still possible in this case for > the handler to switch between the scan chunks, quickly execute the "get" > and then get back to execute the rest of the chunks? > > Also following Ram's suggestion, another thing I found a little confusing > from the UI is that I currently have the RPC handlers to set to 1 on 1 RS. > Still for some reason my UI is showing multiple handlers in a waiting state > even after restarting. > > > > > On Fri, Sep 4, 2015 at 11:18 AM, Matteo Bertozzi <[email protected]> > wrote: > >> scan is chunked in small pieces, it is not a single long operation. >> what fifo vs deadline does is pushing the scan chunk back in the queue >> if you have multiple operation with higher priority. >> >> after N runs of scanner.next() you'll be pushed back following a >> sqrt(numOfNext * weight) >> http://blog.cloudera.com/wp-content/uploads/2014/11/hbase-multi-f2.png >> >> so, the more next you call the further your scan chunk will be delayed. >> but once your scan chunk is in the executor, there is no way (at the >> moment) >> to eject that task, so if you have a single thread and you have an >> hi-priority get >> that get has to wait the scan chunk to be executed. >> >> unless you have tons of requests to fill the queue >> and a super-long scan (or you increase the weight to simulate that) >> you'll probably not be able to notice any difference. >> >> On Fri, Sep 4, 2015 at 9:05 AM, ramkrishna vasudevan < >> [email protected]> wrote: >> >> > Hi Kulkarni >> > >> > Which version of HBase are you using? Just to confirm are you running >> as >> > super user only? I think there are some meta handlers also with which >> > super user calls are handled. >> > >> > From the UI are you able to monitor the handlers and see how many are >> > active and what is the status of it. >> > >> > Regards >> > Ram >> > >> > On Fri, Sep 4, 2015 at 7:42 PM, [email protected] < >> > [email protected]> wrote: >> > >> > > Hello all, >> > > >> > > So I am trying to play around with de-prioritizing long running >> scanners >> > > feature added with HBASE-10993. Essentially I wanted to see how >> > "deadline" >> > > queue prevents contention. This is what my did for my setup. Changed >> the >> > > callqueue type to "fifo" and reduced the number of RPC handlers to 1 >> to >> > > ensure that I have only one RPC queue in place. I am also testing on a >> > > cluster with only one RS to again make sure that there is only one RPC >> > > handler working. I confirmed that my settings are properly in place >> with >> > > the following message in the hmaster logs after I restarted my >> cluster. >> > > >> > > "INFO org.apache.hadoop.hbase.ipc.SimpleRpcScheduler: Using fifo as >> user >> > > call queue, count=1" >> > > >> > > I now kicked off a long running "scan" first and while the scan was >> > running >> > > kicked off another "get". My hope was that with fifo, we should see a >> > > contention and the "get" would essentially hang until the "scan" >> finished >> > > as it would be placed behind the "scan" request in the RPC queue. But >> I >> > see >> > > the gets still return quickly irrespective of whether the scan is >> running >> > > or not. >> > > >> > > Is there something obvious that I am missing here to be able to >> reproduce >> > > the contention? >> > > >> > > Thanks, >> > > Swarnim >> > > >> > >> > > > > -- > Swarnim >
