Re: FlinkKafkaConsumer and Kafka topic/partition change

2016-09-27 Thread Tzu-Li (Gordon) Tai
Hi! This is definitely a planned feature for the Kafka connectors, there’s a JIRA exactly for this [1]. We’re currently going through some blocking tasks to make this happen, I also hope to speed up things over there :) Your observation is correct that the Kaka consumer uses “assign()” instead

FlinkKafkaConsumer and Kafka topic/partition change

2016-09-27 Thread Hironori Ogibayashi
Hello, I want FlinkKafkaConsumer to follow changes in Kafka topic/partition change. This means: - When we add partitions to a topic, we want FlinkKafkaConsumer to start reading added partitions. - We want to specify topics by pattern (e.g accesslog.*), and want FlinkKafkaConsumer to start reading