Hi Florian,

Do you know the set of subtypes in advance? I.e, could you use:

KStream[] branches = stream.branch(predicates);

to split the stream based on the subtypes?

Thanks,
Damian


On Tue, 18 Oct 2016 at 00:43 Matthias J. Sax <matth...@confluent.io> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> Hi,
>
> using DSL you cannot do this. However, if you use Processor API you can.
>
> There are similar question on the mailing list already. For example:
> http://search-hadoop.com/m/uyzND1lghNN1tzbf41&subj=kafka+stream+to+new+t
> opic+based+on+message+key
>
> As we got this request multiple times already, it might be worth
> adding it IMHO. Not sure what the opinion of other is? We should make
> sure that the feature gets accepted before you put a lot of effort in
> it. :)
>
>
> - -Matthias
>
> On 10/17/16 2:10 PM, Florian Hussonnois wrote:
> > Hi All,
> >
> > Currently, it seems not possible with KStream to produce messages
> > to topics which are not known until runtime.
> >
> > For a new project I am evaluating the Kafka Connect / Kafka
> > Streams architecture but without that feature I cannot retain the
> > KStreams API.
> >
> > Our use case is pretty basic. We have xml messages in input of our
> > topology. Each message is splitted into subtypes and formatted in
> > Avro before being sent to a dedicated topic.
> >
> > So the output topics depend of the subtype of each message.
> >
> > I think it would be nice to add methods into the KStream interface
> > to provide such feature.
> >
> > If you think that feature would be usefull I can create a jira and
> > contribute to it. Also, do I need to create a new KIP as this
> > requires changes on a public API ?
> >
> > Thanks,
> >
> -----BEGIN PGP SIGNATURE-----
> Comment: GPGTools - https://gpgtools.org
>
> iQIcBAEBCgAGBQJYBWIhAAoJECnhiMLycopPfTQQAI69Uii5xd8KvaEo/Aeqs0Xw
> AzdPHekdVoHANzo1h45W1x3/lnyeMU/n2v09Agsz46cxb+Xbz9NOKGqT3v9Ye0Ic
> Eyl5yib1B6sWr41rGuYmwDH8zBoC8dPfGZiWhfXL4Sypey3RWzQlVAUWg8Ob4tqF
> rFeubMjWp7yopKRe/7n//JHF029hVK/ePk1vdEsI+2lBI4N7q9ONT/1wKkeCAtdd
> CCkI2WP/WbHzCcUVmOL41KoqgQFnmrH7BtLH67jumzEIR16H+ZenGZmS1uzde56E
> 9mEsl4wmAvfB5GJu6y7JnS7FnQotw7pV7ZneQrA2q8eCZHZqs2fkXf+6ZJNYRir+
> rysqt8wJG69ZN9bSNO1Q6/fNbRiSjYi0I7JnzkErP6scfDKlf3bWzlw6Ejc0+iUr
> Cd0x2m/RlCepVleMT0UZNDlJd0Ml9Q77npP1lyntHVYHjVvtZLdlB5BQYdTMAx3N
> KCLZ+WkY2CBKcwh/KuMr9kW2eWSxH89JSwEule+1bN4vSKyBA6vtrwDoshf6N23g
> dEhTiY5NsgkvAe1JEK6d7PLN2Tq1Tq4OJNoP8PZlqW+YSFl41klQUblo8yT1jSlF
> iCyQS4rgNRabjBs1iZnZNoZ5eodoJMcUyWPhHUYHne+MXuSr1cNNGeNcbS5W0UyE
> dPCe2IiY4zErzxW/Mjmw
> =4DpY
> -----END PGP SIGNATURE-----
>

Reply via email to