Re: Stale TopicMetadata

2013-07-11 Thread Sriram Subramanian
We need to improve how the metadata caching works in kafka. Currently, we have multiple places where we send the updated metadata to the individual brokers from the controller when the state of the metadata changes. This is hard to track. What we need to implement is to let the metadata structure i

Re: Stale TopicMetadata

2013-07-11 Thread Colin Blower
Hm... the cache may explain some odd behavior I was seeing in our cluster yesterday. The zookeeper information for which nodes were In Sync Replicas was different that the data I received from a metadata request response. Zookeeper said two nodes were ISR and the metadata response said only the le

Re: Stale TopicMetadata

2013-07-11 Thread Vinicius Carvalho
Thanks Jun, done. I've created KAFKA-972 issue for that. Regards On Thu, Jul 11, 2013 at 1:16 AM, Jun Rao wrote: > That's actually not expected. We should only return live brokers to the > client. It seems that we never clear the live broker cache in the brokers. > This is a bug. Could you fil

Re: Stale TopicMetadata

2013-07-10 Thread Jun Rao
That's actually not expected. We should only return live brokers to the client. It seems that we never clear the live broker cache in the brokers. This is a bug. Could you file a jira? Thanks, Jun On Wed, Jul 10, 2013 at 8:52 AM, Vinicius Carvalho < viniciusccarva...@gmail.com> wrote: > Hi the