Hi Bowen, To add one more point, the main reason I hope you can provide KafkaMetadataServiceImpl is that I reviewed your PR : https://github.com/apache/flink-connector-kafka/pull/212. It seems there are no IT cases (integration test cases) included. I still don’t fully understand how it works, and I feel a bit uneasy about using it directly in production without clearer verification or guidance.
Best, Hongshun On Fri, Jan 16, 2026 at 10:37 AM Hongshun Wang <[email protected]> wrote: > Hi Bowen， > > In FLIP-515 Dynamic Kafka Sink [1] and FLIP-246: Dynamic Kafka Source [2], > I have always been unclear about what KafkaMetadataServiceImpl is. Is there > an implementation that integrates with the open-source version of Kafka? > Even if there is no open-source implementation, are there any public APIs > available that could guide us on how to integrate with the Metadata > Service? Unfortunately, no one has provided me with an answer so far. Now > that you are revisiting this topic, I hope you can clarify it for me. > > Best， > Hongshun > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-515:+Dynamic+Kafka+Sink > [2] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=217389320 > > On Fri, Jan 16, 2026 at 4:33 AM Bowen Li <[email protected]> wrote: > >> Hi everyone, >> >> I'm kicking off a workstream to enhance and productionize Flink >> DynamicKafkaSource, across areas of core imp, datastream/table/python >> APIs, >> etc >> >> A few related tickets and PRs >> >> - Support per-cluster offset in Dynamic Kafka >> https://issues.apache.org/jira/browse/FLINK-38876 >> <https://issues.apache.org/jira/browse/FLINK-38876> >> - create Dynamic Kafka Source for pyflink >> https://issues.apache.org/jira/browse/FLINK-38530 >> - create Table source and factory for DynamicKafkaSource in Flink Table >> API >> https://issues.apache.org/jira/browse/FLINK-38920 >> >> Dian Fu has generously offered help to review python changes. If you're >> interested in contributing to the workstream, please let me know and feel >> free to jump in on any of the tickets or review PRs. >> >> Best, >> Bowen >> >
