[
https://issues.apache.org/jira/browse/ZOOKEEPER-1504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13553535#comment-13553535
]
Jay Shrauner commented on ZOOKEEPER-1504:
-----------------------------------------
If you want to reconfigure the running AcceptThread, add a new API to the
AcceptThread so you can signal it to close the old listen socket and open a new
one on a new port. Add a new routine "requestReconfig(int port)" that toggles a
boolean ("reconfig=true") and call wakeup() on the AcceptThread. Its main loop
currently only checks for pending new connections. Add so it also checks
whether its been asked to reconfigure itself. If it has, call a new routine
"reconfigure()" that closes the old socket and opens a new one--basically the
same code you had before for the opening of the new one. If you look at the
selector threads, they do something similar, but much more complicated--they
have a pair of message queues as well as the selector. You don't need an entire
queue here.
The other option is to toggle a boolean that tells the AcceptThread not to take
the rest of the socket factory down with it when it shuts down. Then you could
tell one thread to shutdown and fire up another, as Thawan says. The code
should, I believe, with a little modification support multiple simultaneous
listen sockets by creating an AcceptThread per listen socket but using the same
set of selector threads across all of them. Maybe instead of a boolean, add a
factor member variable AtomicInteger that counts the number of active
AcceptThreads, and trigger the factory shutdown only when that count hits 0?
Then for your reconfigure spin up the new AcceptThread/port first before
shutting down the old one. I think I lean towards this latter solution simply
because the work to support your reconfigure would be the same as for
supporting multiple listen sockets, which might be kind of nice to have.
> Multi-thread NIOServerCnxn
> --------------------------
>
> Key: ZOOKEEPER-1504
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1504
> Project: ZooKeeper
> Issue Type: Improvement
> Components: server
> Affects Versions: 3.4.3, 3.4.4, 3.5.0
> Reporter: Jay Shrauner
> Assignee: Jay Shrauner
> Labels: performance, scaling
> Fix For: 3.5.0
>
> Attachments: ZOOKEEPER-1504.patch, ZOOKEEPER-1504.patch,
> ZOOKEEPER-1504.patch, ZOOKEEPER-1504.patch, ZOOKEEPER-1504.patch,
> ZOOKEEPER-1504.patch
>
>
> NIOServerCnxnFactory is single threaded, which doesn't scale well to large
> numbers of clients. This is particularly noticeable when thousands of clients
> connect. I propose multi-threading this code as follows:
> - 1 acceptor thread, for accepting new connections
> - 1-N selector threads
> - 0-M I/O worker threads
> Numbers of threads are configurable, with defaults scaling according to
> number of cores. Communication with the selector threads is handled via
> LinkedBlockingQueues, and connections are permanently assigned to a
> particular selector thread so that all potentially blocking SelectionKey
> operations can be performed solely by the selector thread. An ExecutorService
> is used for the worker threads.
> On a 32 core machine running Linux 2.6.38, achieved best performance with 4
> selector threads and 64 worker threads for a 70% +/- 5% improvement in
> throughput.
> This patch incorporates and supersedes the patches for
> https://issues.apache.org/jira/browse/ZOOKEEPER-517
> https://issues.apache.org/jira/browse/ZOOKEEPER-1444
> New classes introduced in this patch are:
> - ExpiryQueue (from ZOOKEEPER-1444): factor out the logic from
> SessionTrackerImpl used to expire sessions so that the same logic can be used
> to expire connections
> - RateLogger (from ZOOKEEPER-517): rate limit error message logging,
> currently only used to throttle rate of logging "out of file descriptors"
> errors
> - WorkerService (also in ZOOKEEPER-1505): ExecutorService wrapper that
> makes worker threads daemon threads and names then in an easily debuggable
> manner. Supports assignable threads (as used by CommitProcessor) and
> non-assignable threads (as used here).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
