[
https://issues.apache.org/jira/browse/KAFKA-335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Ye updated KAFKA-335:
--------------------------
Attachment: kafka_controller_v2.patch
1. KafkaController:
1.1 Typo satup
-----> fixed
1.2 Sometimes, multiple new lines are added between methods. Can we make this
consistent?
-----> fixed
1.3 It's probably better to create a separate class ControllerChannelManager
that manages all queues and RequestSendThreads. ControllerChannelManager will
support the following API:
def sendRequest(brokerId : Int, request : RequestOrResponse, callback:
(RequestOrResponse) => Unit = null)
def addBroker(brokerId: Int, broker: Broker)
def removeBroker(brokerId: Int)
-----> done
1.4 BrokerChangeListener.handleChildChange(): can we rename curChilds to
javaCurChildren and scalaCurChildren to curChildren since the code is written
in scala?
-----> done
1.5 We don't need to fill in the content of the listener in this jira. Just add
a TODO comment.
-----> the brokerListener is needed
2. RequestSendThread:
2.1 run(): We can't shutdown this thread if no request is added to the queue
since queue.take() is blocking. We need to do something like
ReplicaFetcherThread by interrupting the thread to make the thread shutdownable.
-----> done
3. KafkaZookeeper: It seems to me that it's better to move the controller
election logic to KafkaController. On startup, KafkaController checks if it can
become the controller. If yes, it registers the listeners on brokers and
topics. Otherwise, it registers a listener on the controller path for
controller failover. This way, all logic related to controller is self
contained in KafkaController.
Seems that's not applicable, talk in person
4. BlockingChannel:
4.1 useDefaultBufferSize => UseDefaultBufferSize
-----> fixed
4.2 remove new lines added after the constructor
-----> fixed
4.3 Today, we have logic in SyncProducer and SimpleConsumer to reconnect if
there is any socket IO exception while sending/receiving data. We probably need
something like that in RequestSendThread too. Instead of duplicating this
logic. It's probably better to move this logic to BlockingChannel. I was
thinking that instead of have a send and a receive API, BlockingChannel can
just support 1 sendAndReceive api and implements the reconnect logic for socket
exception.
-----> Let me do it separately, is it ok?
5. ZkUtils: getController and getTopicPartitionLeaderAndISR are not used
anywhere.
-----> removed in the patch
> Implement an embedded controller
> --------------------------------
>
> Key: KAFKA-335
> URL: https://issues.apache.org/jira/browse/KAFKA-335
> Project: Kafka
> Issue Type: Sub-task
> Components: core
> Affects Versions: 0.8
> Reporter: Jun Rao
> Assignee: Yang Ye
> Fix For: 0.8
>
> Attachments: kafka_controller_v1.patch, kafka_controller_v2.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This ticket will implement a controller as described in
> https://cwiki.apache.org/confluence/display/KAFKA/kafka+Detailed+Replication+Design+V3
> This includes creating the controller path, setting up necessary watchers
> (e.g, Broker path and TopicPath), and failover the controller.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
