[
https://issues.apache.org/jira/browse/HBASE-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634653#comment-13634653
]
Jeffrey Zhong commented on HBASE-8368:
--------------------------------------
{quote}
have a thread pool to process events concurrently, and make sure the events
generated on the same znode are always processed in order.
{quote}
I like the idea. Basically, We can have a thread pool with max threads = number
of listeners. Each thread can take all events from a queue at a time. Since the
number of listeners is in 10s(we currently have 9 listeners), In worst cases,
even we have notifications from all queues, it won't cause many context switch.
In the first release, we just provide parallelism for different listeners.
Later we could potentially process region un-assigned notifications
concurrently if those notifications are for different regions(while this needs
more attention) to really scale notification handling up.
> Improve ZK notification handling in Master
> ------------------------------------------
>
> Key: HBASE-8368
> URL: https://issues.apache.org/jira/browse/HBASE-8368
> Project: HBase
> Issue Type: Improvement
> Components: master
> Affects Versions: 0.95.0
> Reporter: Jeffrey Zhong
> Assignee: Jeffrey Zhong
> Fix For: 0.98.0
>
>
> In ZooKeeperWatcher, we handle ZK notifications in a single queue(except AM)
> and each listener is handling notifications synchronously.
> This cause potential performance issue if Master is handling an event which
> may jam the queue so that ZooKeeperWatcher won't get new notifications in
> time. For example, we have several region un-assigned notifications in
> pipeline and logsplitting or SSH handling etc handling will be delayed.
> I'm proposing:
> During registerListeners, we can let a user to choose which queue current
> listener want to register. For example,
> registerListener(QueueName, ZNodePathPrefix, listener);
> Such as:
> registerListener("unsigned_regions", "/hbase/unassigned", AMlistener);
> registerListener("logSplitting", "/hbase/splitlog",
> LogSplittingManagerlistener);
> ...
> For each queue, we use a single thread(consumer) to process events in order.
> The ZNodePathPrefix is to pre-filter out notifications that current listener
> is only interested in.
> For listeners without specifying a queue name, the listener will be put on a
> new "default" queue.
> Please let me know if you have better ideas!
> Thanks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
