[DISCUSS] Should we support a service to manage all deltastreamer jobs?

2023-06-14 Thread 孔维
Hi, team, Background: More and more hudi accesses use deltastreamer, resulting in a large number of deltastreamer jobs that need to be managed. In our company, we also manage a large number of deltastreamer jobs by ourselves, and there is a lot of operation and maintenance management and

Re: [DISCUSS] Should we support a service to manage all deltastreamer jobs?

2023-06-14 Thread Pratyaksh Sharma
Hi, Personally I am in favour of creating such a UI where monitoring and managing configurations is just a click away. That makes life a lot easier for users. So +1 on the proposal. I remember the work for it had started long back around 2019. You can check this RFC

Re: [DISCUSS] Hudi Reverse Streamer

2023-06-14 Thread Nicolas Paris
Hi any rfc/ongoing efforts on the reverse delta streamer ? We have a use case to do hudi => Kafka and would enjoy building a more general tool. However we need a rfc basis to start some effort in the right way On April 12, 2023 3:08:22 AM UTC, Vinoth Chandar wrote: >Cool. lets draw up a RFC

Re: [DISCUSS] Hudi Reverse Streamer

2023-06-14 Thread Rajesh Mahindra
Great. We also need it for use cases of loading data into warehouses, and would love to help. On Wed, Jun 14, 2023 at 9:06 AM Pratyaksh Sharma wrote: > Hi, > > I missed this email earlier. Sure let me start an RFC this week and we can > take it from there. > > On Wed, Jun 14, 2023 at 9:20 PM

Re: [DISCUSS] Hudi Reverse Streamer

2023-06-14 Thread Pratyaksh Sharma
Hi, I missed this email earlier. Sure let me start an RFC this week and we can take it from there. On Wed, Jun 14, 2023 at 9:20 PM Nicolas Paris wrote: > Hi any rfc/ongoing efforts on the reverse delta streamer ? We have a use > case to do hudi => Kafka and would enjoy building a more general