https://issues.apache.org/jira/browse/HADOOP-3856
I have started work on this. The plan is to use selectable channels
(set to blocking mode), a work queue, and a pool of worker threads
to handle xceiver connections. The hope is much fewer actual threads
can do equivalent work as 2048 or 4096 dedicated per connection
xceiver threads.
However, it may be necessary to go fully async, if HBase load
patterns look more like a large number of slowly reading or writing
clients as opposed to simply a flood of requests that complete
relatively quickly.
I have looked at some async I/O toolkits:
- Grizzly: has a lot of stuff that is not needed
- MINA: plus that it is an ASF technology
- Netty: LGPL, cannot use
- xSocket: LGPL, cannot use
Probably if it's necessary to go fully async, I'll look if MINA can
help, if there's some compelling benefit. One important goal is to
get the changes committed upstream: I'd prefer not to add an
external dependency, even an ASF project, to complicate the issue.
I welcome comments or suggestions on this approach.
Thanks,
- Andy
