A topic.metadata.refresh.interval.ms of 10 mins means that the producer will take 10 mins to detect new partitions. So newly added or reassigned partitions might not get data for 10 mins. In general, if you're still at prototyping stages, I'd recommend using the new producer available on kafka trunk (org.apache.kafka.clients.producer.KafkaProducer). It has better performance and APIs.
On Thu, Oct 16, 2014 at 3:07 AM, Mungeol Heo <mungeol....@gmail.com> wrote: > Hi, > > I have a question about 'topic.metadata.refresh.interval.ms' > configuration. > As I know, the default value of it is 10 minutes. > Does it means that producer will change the partition at every 10 minutes? > What I am experiencing is producer does not change to another > partition at every 10 minutes. > Sometime, It never changed during the process which costs about 25 minutes. > I also changed the value of it to 1 minute for testing. > It looks like working well at first time. > However, same problem happens start from second test. > Sometime, it takes more than 10 minutes to change the partition even > if I set the value as 1 minute. > Am i missing something? > Any help will be great. > > Thanks. > > - Mungeol >
