Hey, I have few doubts about HybridSource, it says:
> To arrange multiple sources in a HybridSource, all sources except the > last one need to be bounded. Therefore, the sources typically need to be > assigned a start and end position. Both sources that I use in the job are unbounded, (file source and Kafka source both get continuous updates). Further schema of both sources is different since they give different data points. So wanted to clarify that further you are suggesting converting Kafka source to hybrid source(file source and Kafka source) and update state based on that and keep file source as it is. *Akshay Agarwal* On Fri, Oct 15, 2021 at 2:06 PM Akshay Agarwal <akshay.agar...@grofers.com> wrote: > Nope, I haven't gone through that, will take a look, thanks for the prompt > reply. > > *Akshay Agarwal* > > > > On Fri, Oct 15, 2021 at 2:00 PM Martijn Visser <mart...@ververica.com> > wrote: > >> Hi, >> >> Have you checked out the Hybrid Source? [1] >> >> Thanks, Martijn >> >> [1] >> https://ci.apache.org/projects/flink/flink-docs-master/docs/connectors/datastream/hybridsource/ >> >> On Fri, 15 Oct 2021 at 10:22, Akshay Agarwal <akshay.agar...@grofers.com> >> wrote: >> >>> Hi Team, >>> >>> I have a streaming job that creates an enriched stream from streams of 2 >>> different sources like one from files in object storage and another one >>> Kafka. But since both of these are from different sources, events from them >>> are read at different moments based on their performance. For example >>> events from Kafka arrives first than from file storage events and even >>> though I have used event watermarks, thus at the restart of jobs, enriched >>> events are wrong and it gets corrected after file events start arriving. I >>> tried to search for setting up synchronization between different sources in >>> flink but did not found any blog/material. It might be a noob question, but >>> if you guys have built something around this, could you let me know. >>> >>> Regards >>> *Akshay Agarwal* >>> >>> [image: https://grofers.com] <https://grofers.com> >> >> -- <https://grofers.com>
