Hi there, Working on the streaming processing latency time based on timestamps from Kafka, I have two quick general questions triggered by looking at the kafka stage change log file:
(a) the partition state change from OfflineReplica state *to OnlinePartition *state seems to take more than 20 sec. Would it mean that an incoming message/event into kafka would need to go thru all these state transitions (see below) to become ready for consumer (in this case, after 20+ sec)? (b) For which stage would be the kafka time stampe for? Any help/clarification would be very much appreciated! Thanks ... Peter [2018-06-08 *15:34:36*,518] TRACE Controller 0 epoch 53 changed state of replica 0 for partition [events,79] from *ReplicaDeletionIneligible *to OfflineReplica (state.change.logger) [2018-06-08 15:34:36,945] TRACE Controller 0 epoch 53 changed state of replica 0 for partition [events,79] from OfflineReplica to OnlineReplica (state.change.logger) [2018-06-08 15:34:37,025] TRACE Controller 0 epoch 53 sending become-follower LeaderAndIsr request (Leader:-1,ISR:,LeaderEpoch:1,ControllerEpoch:53) to broker 0 for partition [events,79] (state.change.logger) [2018-06-08 15:34:37,079] TRACE Broker 0 received LeaderAndIsr request PartitionState(controllerEpoch=53, leader=-1, leaderEpoch=1, isr=[], zkVersion=1, replicas=[0]) correlation id 1 from controller 0 epoch 53 for partition [events,79] (state.change.logger) [2018-06-08 15:34:38,481] TRACE Controller 0 epoch 53 changed state of replica 0 for partition [events,79] from OnlineReplica to OfflineReplica (state.change.logger) [2018-06-08 15:34:38,588] TRACE Controller 0 epoch 53 changed state of replica 0 for partition [events,79] from OfflineReplica to ReplicaDeletionStarted (state.change.logger) [2018-06-08 15:34:39,427] TRACE Controller 0 epoch 53 changed state of replica 0 for partition [events,79] from ReplicaDeletionStarted to ReplicaDeletionSuccessful (state.change.logger) [2018-06-08 15:34:39,560] TRACE Controller 0 epoch 53 changed state of replica 0 for partition [events,79] from ReplicaDeletionSuccessful to NonExistentReplica (state.change.logger) [2018-06-08 15:34:39,564] TRACE Controller 0 epoch 53 changed partition [events,79] state from OfflinePartition to OfflinePartition (state.change.logger) [2018-06-08 15:34:39,571] TRACE Controller 0 epoch 53 changed partition [events,79] state from OfflinePartition to NonExistentPartition (state.change.logger) [2018-06-08 15:35:01,893] TRACE Controller 0 epoch 53 changed partition [events,79] state from NonExistentPartition to NewPartition with assigned replicas 0 (state.change.logger) [2018-06-08 15:35:01,960] TRACE Controller 0 epoch 53 changed state of replica 0 for partition [events,79] from NonExistentReplica to NewReplica (state.change.logger) [2018-06-08 *15:35:02,02*6] TRACE Controller 0 epoch 53 changed partition [events,79] from *NewPartition *to *OnlinePartition *with leader 0 (state.change.logger) [2018-06-08 15:35:02,207] TRACE Controller 0 epoch 53 sending become-leader LeaderAndIsr request (Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:53) to broker 0 for partition [events,79] (state.change.logger) [2018-06-08 15:35:02,219] TRACE Broker 0 received LeaderAndIsr request PartitionState(controllerEpoch=53, leader=0, leaderEpoch=0, isr=[0], zkVersion=0, replicas=[0]) correlation id 88 from controller 0 epoch 53 for partition [events,79] (state.change.logger)
