+1. Happy to see DeltaStreamer becomes more and more powerful. Also, we need to pay some attention to the layout and organization of these connectors as more and more data sources introduced to HUDI like vinoyang suggested.
Best, Leesf Bhavani Sudha Saktheeswaran <[email protected]> 于2019年9月22日周日 下午12:18写道： > +1 to adding more connectors to DeltStreamer and making them as much > pluggable modules as possible like Vino Yang suggested. > > > On Sat, Sep 21, 2019 at 7:12 PM vino yang <[email protected]> wrote: > > > + 1 to introduce these connectors. It's nice to see that Hudi's ecosystem > > is growing. As Hudi connects to more and more systems, it is necessary to > > introduce separate modules to place these connectors. This can lead to > > module relayout or code refactoring. Of course, all this needs to be > > discussed in more depth. Best， Vino On 09/21/2019 18:59, Vinay Patil > wrote: > > Hi Taher, Basically this can be proposal to support Kinesis and DynamoDb > > stream support can be enabled by reusing this source code. Flink has > > provided support for DynamoDb Streams by reusing Kinesis Streams classes. > > Regards, Vinay Patil On Sat, Sep 21, 2019 at 4:26 PM Taher Koitawala < > > [email protected]> wrote: > That would be a great addition Vinay. How > > about adding Kinesis as well? > > Regards, > Taher Koitawala > > On Sat, > > Sep 21, 2019, 4:20 PM Vinay Patil <[email protected]> wrote: > > > > > Hi Team, > > > > The DynamoDb streams contains the CDC data when enabled > on > > a DynamoDb > > table, we can add a source for DeltaStreamer which will > > enable us to read > > this data and write it back either to Hudi dataset > or > > to another sink. > > > > > > Thoughts on adding this support in Hudi ? > > > > > > > > > Regards, > > Vinay Patil > > > >
