+1 to adding more connectors to DeltStreamer and making them as much pluggable modules as possible like Vino Yang suggested.
On Sat, Sep 21, 2019 at 7:12 PM vino yang <[email protected]> wrote: > + 1 to introduce these connectors. It's nice to see that Hudi's ecosystem > is growing. As Hudi connects to more and more systems, it is necessary to > introduce separate modules to place these connectors. This can lead to > module relayout or code refactoring. Of course, all this needs to be > discussed in more depth. Best， Vino On 09/21/2019 18:59, Vinay Patil wrote: > Hi Taher, Basically this can be proposal to support Kinesis and DynamoDb > stream support can be enabled by reusing this source code. Flink has > provided support for DynamoDb Streams by reusing Kinesis Streams classes. > Regards, Vinay Patil On Sat, Sep 21, 2019 at 4:26 PM Taher Koitawala < > [email protected]> wrote: > That would be a great addition Vinay. How > about adding Kinesis as well? > > Regards, > Taher Koitawala > > On Sat, > Sep 21, 2019, 4:20 PM Vinay Patil <[email protected]> wrote: > > > > Hi Team, > > > > The DynamoDb streams contains the CDC data when enabled on > a DynamoDb > > table, we can add a source for DeltaStreamer which will > enable us to read > > this data and write it back either to Hudi dataset or > to another sink. > > > > > > Thoughts on adding this support in Hudi ? > > > > > > > Regards, > > Vinay Patil > > >
