Cool. lets draw up a RFC for this? @pratyaksh - do you want to start one, given you expressed interest?
On Mon, Apr 10, 2023 at 7:32 PM Léo Biscassi <leo.bisca...@gmail.com> wrote: > +1 > This would be great! > > Cheers, > > On Mon, Apr 3, 2023 at 3:00 PM Pratyaksh Sharma <pratyaks...@gmail.com> > wrote: > > > Hi Vinoth, > > > > I am aligned with the first reason that you mentioned. Better to have a > > separate tool to take care of this. > > > > On Mon, Apr 3, 2023 at 9:01 PM Vinoth Chandar < > > mail.vinoth.chan...@gmail.com> > > wrote: > > > > > +1 > > > > > > I was thinking that we add a new utility and NOT extend DeltaStreamer > by > > > adding a Sink interface, for the following reasons > > > > > > - It will make it look like a generic Source => Sink ETL tool, which is > > > actually not our intention to support on Hudi. There are plenty of good > > > tools for that out there. > > > - the config management can get bit hard to understand, since we > overload > > > ingest and reverse ETL into a single tool. So break it off at use-case > > > level? > > > > > > Thoughts? > > > > > > David: PMC does not have control over that. Please see unsubscribe > > > instructions here. https://hudi.apache.org/community/get-involved > > > Love to keep this thread about reverse streamer discussion. So kindly > > fork > > > another thread if you want to discuss unsubscribing. > > > > > > On Fri, Mar 31, 2023 at 1:47 AM Davidiam <david.rosa...@gmail.com> > > wrote: > > > > > > > Hello Vinoth, > > > > > > > > Can you please unsubscribe me? I have been trying to unsubscribe for > > > > months without success. > > > > > > > > Kind Regards, > > > > David > > > > > > > > Sent from Outlook for Android<https://aka.ms/AAb9ysg> > > > > ________________________________ > > > > From: Vinoth Chandar <vin...@apache.org> > > > > Sent: Friday, March 31, 2023 5:09:52 AM > > > > To: dev <dev@hudi.apache.org> > > > > Subject: [DISCUSS] Hudi Reverse Streamer > > > > > > > > Hi all, > > > > > > > > Any interest in building a reverse streaming tool, that does the > > reverse > > > of > > > > what the DeltaStreamer tool does? It will read Hudi table > incrementally > > > > (only source) and write out the data to a variety of sinks - Kafka, > > JDBC > > > > Databases, DFS. > > > > > > > > This has come up many times with data warehouse users. Often times, > > they > > > > want to use Hudi to speed up or reduce costs on their data ingestion > > and > > > > ETL (using Spark/Flink), but want to move the derived data back into > a > > > data > > > > warehouse or an operational database for serving. > > > > > > > > What do you all think? > > > > > > > > Thanks > > > > Vinoth > > > > > > > > > > > > -- > *Léo Biscassi* > Blog - https://leobiscassi.com > > - >
