Re: KStreams / add support for sink processor with dynamic topics

Florian, I'd also have a follow-up question: > Actually, the set of subtypes is not known in advance this is why I need to > compute output topics from messages. So the branch method is of any help > in my context. For a number of reasons we recommend pre-creating input, intermediate, and

Re: KStreams / add support for sink processor with dynamic topics

We can consider adding this feature to with a StreamsAdminClient that we are adding as part of KAFKA-4060. However, I'm still not sure if it should be added on the DSL layer or on the Processor API layer. Florian, what do you mean that the Processor is not "completely safe"? Do you mean not

Re: KStreams / add support for sink processor with dynamic topics

Thank you Matthias for your answers. The mailing list that you linked shows a solution using the Processor API. Actually, the set of subtypes is not known in advance this is why I need to compute output topics from messages. So the branch method is of any help in my context. I think, this

Re: KStreams / add support for sink processor with dynamic topics

Hi Florian, Do you know the set of subtypes in advance? I.e, could you use: KStream[] branches = stream.branch(predicates); to split the stream based on the subtypes? Thanks, Damian On Tue, 18 Oct 2016 at 00:43 Matthias J. Sax wrote: > -BEGIN PGP SIGNED

Re: KStreams / add support for sink processor with dynamic topics

-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi, using DSL you cannot do this. However, if you use Processor API you can. There are similar question on the mailing list already. For example: http://search-hadoop.com/m/uyzND1lghNN1tzbf41=kafka+stream+to+new+t opic+based+on+message+key As we

KStreams / add support for sink processor with dynamic topics

Hi All, Currently, it seems not possible with KStream to produce messages to topics which are not known until runtime. For a new project I am evaluating the Kafka Connect / Kafka Streams architecture but without that feature I cannot retain the KStreams API. Our use case is pretty basic. We