[
https://issues.apache.org/jira/browse/KAFKA-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14513540#comment-14513540
]
Gwen Shapira commented on KAFKA-2139:
-------------------------------------
Thanks.
Good point about separate port not solving the issue :)
Can you elaborate on why do you think its a good idea to reuse NetworkClient in
SocketServer (vs. just the Selector)? Are there specific features there that
you need? Most of the NetworkClient code seemed very specific to clients, but
not to a server (things like leastLoadedNode especially). The generic stuff
(i.e. Poll) also has tons of client specific code (like handling Metadata
fetches). It seems like a very awkward fit for SocketServer. Selector on the
other hand, seemed like the almost perfect abstraction (except for being
non-threadsafe and not having its own queues).
It is possible to create another layer in-between on o.a.k.common.network, but
since there's just one server - why not handle the server specific code in
SocketServer using o.a.k.common.network.Selector?
TL;DR;
I'll try to upload a new patch to KAFKA-1928 shortly so you can see what I mean.
> Add a separate controller messge queue with higher priority on broker side
> ---------------------------------------------------------------------------
>
> Key: KAFKA-2139
> URL: https://issues.apache.org/jira/browse/KAFKA-2139
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jiangjie Qin
> Assignee: Jiangjie Qin
>
> This ticket is supposed to be working together with KAFKA-2029.
> There are two issues with current controller to broker messages.
> 1. On the controller side the message are sent without synchronization.
> 2. On broker side the controller messages share the same queue as client
> messages.
> The problem here is that brokers process the controller messages for the same
> partition at different times and the variation could be big. This causes
> unnecessary data loss and prolong the preferred leader election / controlled
> shutdown/ partition reassignment, etc.
> KAFKA-2029 was trying to add a boundary between messages for different
> partitions. For example, before leader migration for previous partition
> finishes, the leader migration for next partition won't begin.
> This ticket is trying to let broker process controller messages faster. So
> the idea is have separate queue to hold controller messages, if there are
> controller messages, KafkaApi thread will first take care of those messages,
> otherwise it will proceed messages from clients.
> Those two tickets are not ultimate solution to current controller problems,
> but just mitigate them with minor code changes. Moving forward, we still need
> to think about rewriting controller in a cleaner way.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
