+1 there are RFCs on table management services, but not specific to deltastreamer itself.
Are you proposing building something specific to that? On Wed, Jun 14, 2023 at 8:26 AM Pratyaksh Sharma <pratyaks...@gmail.com> wrote: > Hi, > > Personally I am in favour of creating such a UI where monitoring and > managing configurations is just a click away. That makes life a lot easier > for users. So +1 on the proposal. > > I remember the work for it had started long back around 2019. You can check > this RFC > < > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=130027233 > > > for your reference. I am not sure why this work could not continue though. > > On Wed, Jun 14, 2023 at 4:28 PM 孔维 <18701146...@163.com> wrote: > > > Hi, team, > > > > > > Background: > > More and more hudi accesses use deltastreamer, resulting in a large > number > > of deltastreamer jobs that need to be managed. In our company, we also > > manage a large number of deltastreamer jobs by ourselves, and there is a > > lot of operation and maintenance management and monitoring work. > > If we can provide such a deltastreamer service to create, manage, and > > monitor all tasks in a unified manner, it can greatly reduce the > management > > pressure of deltastreamer, and at the same time lower the threshold for > > using deltastreamer, which is conducive to the promotion and use of > > deltastreamer. > > At the same time, considering that deltastreamer already supports > > configuration hot update capability [ > > https://github.com/apache/hudi/pull/8807], we can offer configuration > hot > > update capability based on the feature, and make configuration changes > > without restarting the job. > > > > > > We hope to provide: > > Provides a web UI to support creation, management and monitoring of > > deltastreamer tasks > > Using configuration hot update capability to provide timely configuration > > change capability > > > > > > I don't know whether such a service is in line with the evolution of the > > community, and I hope to receive your reply! > > > > > > Best Regards >
