this topic foo is created by auto creation at Sep 3. I looked at the old
log files on controller broker 5. it has some errors regarding partition 1
and 2 when the topic is auto created. who is creating/updating IsrLeader
record?
errors regarding partition 1
noticed one important thing. topic foo's partition 1 and 2 have empty .log
file on replicas. I suspect replication doesn't create the partition dir on
broker 0 in this case, which then cause the WARN logs.
On Mon, Sep 8, 2014 at 11:11 PM, Steven Wu stevenz...@gmail.com wrote:
sorry. forgot to
Hmm, that seems to indicate that the leader info is not propagated properly
from the controller to broker 0. In the state-change log of broker 0, do
you see anything related to these 2 partitions when broker 0 comes up?
Thanks,
Jun
On Tue, Sep 9, 2014 at 9:41 AM, Steven Wu stevenz...@gmail.com
nope. sate-change log files only had some warnings regarding other
partitions. nothing related to these two partitions.
2014-09-09 02:54:30,579 WARN kafka.utils.Logging$class:83
[kafka-request-handler-1] [warn] Broker 0 received invalid LeaderAndIsr
request with correlation id 497 from
assuming you are talking about controller log on broker 0, there is nothing
there.
$ less kafka-controller
2014-09-09 01:15:02,600 INFO kafka.utils.Logging$class:68 [main] [info]
[ControllerEpochListener on 0]: Initialized controller epoch to 5 and zk
version 4
2014-09-09 01:15:02,655 INFO
ah. maybe you mean the controller log on leader/controller broker 5. yes. I
do noticed some errors regarding these two partitions.
[2014-09-09 01:10:53,651] ERROR Controller 5 epoch 5 encountered error
while changing partition [foo,2]'s state from New to Online since
LeaderAndIsr path alrea
dy
previous email is from state-change.log. also found this WARN in
controller.log
[2014-09-09 01:10:53,669] WARN [Controller 5]: Cannot remove replica 0 from
ISR of partition [cdn_selection_runtime_data,8] since it is not in the ISR.
Leader = 1 ; ISR = List(1) (kafka.controller.KafkaController)
did a push in cloud. after new instance for broker 0 comes up, I see a lot
of WARNs in log file.
2014-09-09 04:21:09,271 WARN kafka.utils.Logging$class:83
[request-expiration-task] [warn] [KafkaApi-0] Fetch request with
correlation id 51893 from client 1409779957450-6014fc32-0-0 on partition
