[
https://issues.apache.org/jira/browse/FLINK-22748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17886904#comment-17886904
]
Benoît Paris commented on FLINK-22748:
--------------------------------------
Well, I'd like to pause and congratulate all the contributors!
First [~twalthr] , [~jark] for the issue management and reviews,
[~nicholasjiang] for the first attempt, then [~klam-shop] for picking up the
feature again; and finally [~arvid] for the numerous reviews helping this
getting merged.
Here we are 3 years later; and I can finally enjoy this quite niche feature.
For free thanks to you guys.
I'm quite curious as to its use other than the one I have, now. I'll share it:
This is for distributing reactive web apps. Backend publishes messages from a
user to a topic, business rules and event mixing happen in Flink, back it goes
to the backend for serving the emergent results to potentially other users.
Before this patch I had to make backends read all the messages of a single
topic and filter, now they'll register which topic they're going to listen to,
and which user they are serving; and Flink will make the routing happen, making
a part of my system scalable.
So, what's yours at Celeborn, [~nicholasjiang] ? And yours at Shopify
[~klam-shop] ?
> Allow dynamic target topic selection in SQL Kafka sinks
> -------------------------------------------------------
>
> Key: FLINK-22748
> URL: https://issues.apache.org/jira/browse/FLINK-22748
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka, Table SQL / Ecosystem
> Reporter: Timo Walther
> Assignee: Kevin Lam
> Priority: Major
> Labels: pull-request-available, stale-assigned, starter
> Fix For: kafka-3.3.0
>
>
> We should allow to write to different Kafka topics based on some column value
> in SQL.
> The existing implementation can be easily adapted for that. The "target
> topic" would be an additional persisted metadata column in SQL terms. All one
> need to do is to adapt
> DynamicKafkaSerializationSchema
> KafkaDynamicSink
> We should guard this dynamic behavior via a config option and make the topic
> option optional in this case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
