RE: [DISCUSS] Contribution of Multi Cluster Kafka Source

2022-09-02 Thread Ryan van Huuksloot
Hi Mason, First off, thanks for putting this FLIP together! Sorry for the delay. Full disclosure Mason and I chatted a little bit at Flink Forward 2022 but I have tried to capture the questions I had for him then. I'll start the conversation with a few questions: 1. The concept of streamIds is

Re: [DISCUSS] Contribution of Multi Cluster Kafka Source

2022-07-14 Thread Mason Chen
Hi all, Circling back on this--I have created a first draft document in confluence: https://cwiki.apache.org/confluence/display/FLINK/FLIP-246%3A+Multi+Cluster+Kafka+Source . Looking forward to hear all your feedback in this email thread! Best, Mason On Thu, Jun 30, 2022 at 6:57 AM Thomas

Re: [DISCUSS] Contribution of Multi Cluster Kafka Source

2022-06-30 Thread Thomas Weise
Hi Mason, I added mason6345 to the Flink confluence space, you should be able to add a FLIP now. Looking forward to the contribution! Thomas On Thu, Jun 30, 2022 at 9:25 AM Martijn Visser wrote: > > Hi Mason, > > I'm sure there's a PMC (*hint*) out there who can grant you access to > create a

Re: [DISCUSS] Contribution of Multi Cluster Kafka Source

2022-06-30 Thread Martijn Visser
Hi Mason, I'm sure there's a PMC (*hint*) out there who can grant you access to create a FLIP. Looking forward to it, this sounds like an improvement that users are looking forward to. Best regards, Martijn Op di 28 jun. 2022 om 09:21 schreef Mason Chen : > Hi all, > > Thanks for the

Re: [DISCUSS] Contribution of Multi Cluster Kafka Source

2022-06-28 Thread Mason Chen
Hi all, Thanks for the feedback! I'm adding the users, who responded in the user mailing list, to this thread. @Qingsheng - Yes, I would prefer to reuse the existing Kafka connector module. It makes a lot of sense since the dependencies are the same and the implementation can also extend and

RE: Re: [DISCUSS] Contribution of Multi Cluster Kafka Source

2022-06-28 Thread Ryan van Huuksloot
Hi Mason, Thanks for starting this discussion! The proposed Source sounds awesome and we would be interested in taking a look at the source code and evaluating our use cases. We can provide information and review on a potential FLIP based on other use cases. Do you have a fork/branch that you

Re: [DISCUSS] Contribution of Multi Cluster Kafka Source

2022-06-26 Thread Qingsheng Ren
Hi Mason, It sounds like an exciting enhancement to the Kafka source and will benefit a lot of users I believe. Would you prefer to reuse the existing flink-connector-kafka module or create a new one for the new multi-cluster feature? Personally I prefer the former one because users won’t