Adding Nifi dev@ to this thread.
On Wed, Sep 18, 2019 at 10:57 AM Vinoth Chandar <vin...@apache.org> wrote: > Not too familiar wth Nifi myself. Would this still target an use-case like > what pratyaksh mentioned? > For delta streamer specifically, we are moving more and more towards > continuous mode, where > Hudi writing and compaction are amanged by a single long running spark > application. > > Would Nifi also help us manage compactions when working with Hudi > datasource or just writing plain spark Hudi pipelines? > > On 2019/09/18 08:18:44, Taher Koitawala <taher...@gmail.com> wrote: > > That's another way of doing things. I want to know if someone wrote > > something like PutParquet. Which directly can write data to Hudi. AFAIK I > > don't think anyone has. > > > > That will really be powerful. > > > > On Wed, Sep 18, 2019, 1:37 PM Pratyaksh Sharma <pratyaks...@gmail.com> > > wrote: > > > > > Hi Taher, > > > > > > In the initial phase of our CDC pipeline, we were using Hudi with Nifi. > > > Nifi was being used to read Binlog file of mysql and to push that data > to > > > some Kafka topic. This topic was then getting consumed by > DeltaStreamer. So > > > Nifi was indirectly involved in that flow. > > > > > > On Wed, Sep 18, 2019 at 10:29 AM Taher Koitawala <taher...@gmail.com> > > > wrote: > > > > > > > Hi All, > > > > Just wanted to know has anyone tried to write data to Hudi > > > with a > > > > Nifi flow? > > > > > > > > Perhaps may be just a csv file on local to Hudi dataset? If not then > lets > > > > try that! > > > > > > > > Regards, > > > > Taher Koitawala > > > > > > > > > >
