Re: Understanding Kafka controller log

2020-05-20 Thread Amitav Mohanty
Hi Liam, Thank you for the clarification. My use of words was a bit confusing. Let me rephrase it. :) I believe each partition has a leader. This is liable to change in case of any broker going down. What I am interested in it getting logs from the current leader of any partition. For that, I

Re: Understanding Kafka controller log

2020-05-19 Thread Liam Clarke-Hutchinson
Hi Amitav, Brokers only write to controller log when they're the cluster controller. So if you wanted to see what the cluster controller was doing at a given point in time, then yep, you'd want to aggregate all controller.log files to handle controller changes. Typically controller status is very

Understanding Kafka controller log

2020-05-19 Thread Amitav Mohanty
Hi I am trying to understand the right way of viewing controller logs. As the controller logs are written in each broker, do we need to see all of them to know the state of the cluster at any given point in time ? To follow the leader of any given partition, do we need to correlate across