Sure sorry didn't provide a good example. There are two situations where I have thought this feature might be useful. Maybe more...
1. For something like Phoenix, there are situations where you want certain operations for tables / users to always have handlers available to do work. For example any write to an index table should never block. One way to almost guarantee this is to give it its own special set of handlers and allow from a client call to denote that this call is meant for this specific handler pool. 2. Suppose you have a large cluster with 1 really large table. You can't really use regionserver groups and there are clients doing all sorts of operations on this cluster. Map/Reduce jobs (long scans, heavy reduces), processing pipeline, random reads, etc. There are features already to de-prioritize long running scanners and there is rpc throttling and we can split up the handlers into read, write. Currently there is no easy way to say I want to reserve 10 handlers on each regionserver for the processing pipeline and from the client you can pass something along that would tell the server to use this special handler pool. Also Stack, I saw your TODO and I believe we could get rid of the AnnotatinoReadingPriorityFunction. We can talk about design if folks are interested. Thanks rahul > On Apr 21, 2016, at 12:05 AM, Mikhail Antonov <[email protected]> wrote: > > This is interesting idea. Sorry if I missed some context - what's the > primary incentive here? What's examples of those categorized thread pools? > > Sounds intersecting a bit with HBASE-15136 > <https://issues.apache.org/jira/browse/HBASE-15136> (deadline scheduling > for RPC requests) in the area of rpc prioritizing. > > -Mikhail > >> On Wed, Apr 20, 2016 at 11:21 PM, Stack <[email protected]> wrote: >> >> On Wed, Apr 20, 2016 at 1:47 PM, rahul gidwani <[email protected]> >> wrote: >> >>> I was wondering if people would be interested in allowing the client to >>> specify priorities? I really think we are good responsible adults and >> wont >>> abuse this feature. :) >>> >>> This would not just be for one particular operation but all operations. >>> I'll make it feature complete. >> Sounds sweet. >> >> RPC passes priority in the header already IIRC. >> >> We could then purge our ugly decompose of the request just to figure what >> it is so we can prioritize based off annotation. >> >> St.Ack >> >> >> >>> As for batch operations prioirites would be at batch level. >>> >>> I know the phoenix guys would really like this feature as it would really >>> help with their indexing work. >> >> >>> Eventually I think it would be nice to get to a point where we can have >>> some sort of configurable reservation system. Where regionservers could >>> have handler groups and we could send a little bit more info with the rpc >>> call to specify the reserved set of handlers they would like to utilize. >>> >>> thanks >>> rahul > > > > -- > Thanks, > Michael Antonov
