Hi Prasanna,
auto.create.topics.enable is only recommended for development clusters and
not in production use cases (as one programming error could potentially
flood the whole broker with a large amount of topics). I have experienced
first hand the mess it makes.
I'd suggest finding a supplemental
I think "auto.create.topics.enable" is enabled by default [1]?
Best,
Jark
[1]: https://kafka.apache.org/documentation/#auto.create.topics.enable
On Mon, 1 Jun 2020 at 19:55, Leonard Xu wrote:
> I think @brat is right, I didn’t know the Kafka property
> 'auto.create.topics.enable’ , you can pa
I think @brat is right, I didn’t know the Kafka property
'auto.create.topics.enable’ , you can pass the property to Kafka Producer, that
should work.
Best,
Leonard Xu
> 在 2020年6月1日，18:33，satya brat 写道：
>
> Prasanna,
> You might want to check the kafka broker configs where
> 'auto.create.topi
Prasanna,
You might want to check the kafka broker configs where
'auto.create.topics.enable' helps with creating a new topic whenever a new
message with non existent topic is published.
https://kafka.apache.org/documentation/#brokerconfigs
I am not too sure about pitfalls if any.
On Mon, Jun 1, 2
Hi, kumar
Sorry for missed the original question, I think we can not create topic
dynamically current, creating topic should belong to control flow rather a data
flow, and user may has some custom configurations of the topic from my
understanding. Maybe you need implement the logic of check/cre
Leaonard,
Thanks for the reply and would look into those options.
But as for the original question, could we create a topic dynamically when
required .
Prasanna.
On Mon, Jun 1, 2020 at 2:18 PM Leonard Xu wrote:
> Hi, kumar
>
> Flink support consume/produce from/to multiple kafka topics[1], in
Hi, kumar
Flink support consume/produce from/to multiple kafka topics[1], in your case
you can implement KeyedSerializationSchema(legacy interface) or
KafkaSerializationSchema[2] to make one producer instance support send data to
multiple topics. There is an ITCase you can reference[3].
Best,
