[
https://issues.apache.org/jira/browse/BOOKKEEPER-70?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122684#comment-13122684
]
Ivan Kelly commented on BOOKKEEPER-70:
--------------------------------------
Comments which have been made about this before:
Current, in Hedwig, each topic subscription has its own network channel. Thus
for every topic present on a hub, it uses
up one port for connecting to the subscriber, and one each for subscribing to
that topic in all the other regions. If
we have a 3 region setup, each topic takes up a minimum of 4 ports. Since there
are 64K possible ports, a hub can only
handle 16K topics. This number is less in reality because some ports are system
reserved, others are need for the
publish calls etc. This number goes down further if we have more regions.
The client should be changed to maintain at most one subscription channel to
each hub. Multiple subscriptions can then
be serviced over the same channel. This will allow us to scale independent of
the number of topics but in the number of
hosts we have to connect to.
This will require a couple of server-side changes as well:
i) txn-id of original subscribe request should be sent back with every request,
so that client can decide which
subscriber the message is for.
ii) Currently the server has a map of channel to subscriber, that should be
changed to a multimap.
> reduce or demultiplex subscription connections to a hub server
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-70
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-70
> Project: Bookkeeper
> Issue Type: Improvement
> Components: hedwig-client, hedwig-server
> Reporter: Sijie Guo
>
> now each subscription will establish a connection to hub server, there will
> be too many connections on both client/server sides.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
