CalvinKirs opened a new issue #4878: URL: https://github.com/apache/incubator-dolphinscheduler/issues/4878
1: Loading of plug-ins Worker needs to load all plug-ins when it starts, complete plug-in loading, update (plugin_define table) and other operations (refer to the relevant plug-in loading when AlertSerrver starts. Different services load plug-ins should be different, this needs to specify loading plug-ins Type, including the following registration center or other SPI) The dependency in the worker is more complicated, and the SPI module needs to be disassembled. Some plug-ins rely on too many external jars, such as hadoop. There needs to be a sink, which plug-in needs to be placed in which plug-in. 2: Front-end controls The front-end interface display is still from-create, so all interactions are placed in the back-end. We need to add some complex forms to the original form types, such as linkage effects, event triggers (it is best to implement complex interactions internally, plug-ins Don’t make users too cumbersome to use, otherwise the cost of education will be too high) Participate specifically: dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params The task plug-in currently interacts with the database in two parts: data source and resource center. This is unchanged, spi only saves relevant specific information. The old logic can be used for query and call, and there will be related choices in the form. 3: Expansion interface Extension interface (the user plug-in implements two interfaces that need to be paid attention to. I haven't mentioned the specific code yet, and there may be modifications, but mainly these two pieces) All task plugins need to implement the task execution interface (mainly two blocks: cancel (task cancellation), and submit (task execution)) All task plugins need to implement the relevant interface of the task UI (used for interface display, roughly similar to alert-plugin, api (UiPluginController does not need to be modified, only need to add a type)) We will store the relevant information of the plugin in plugin_define Table. It should be noted that we still have to reduce plugin dependencies as much as possible. 1:插件的装载 Worker在启动的时候需要加载所有插件，完成插件的装载、更新（plugin_define表）等操作（具体参考AlertSerrver启动的时候相关插件加载，不同的服务加载的插件应该是不同的，这块需要指定装载插件的类型，包括后面注册中心或者其他SPI同样） worker中依赖较为复杂，SPI模块需要想办法拆解出去。有的插件依赖了太多的外部jar，比如hadoop。这里需要下沉，具体哪个插件需要就放在哪个插件中去。 2:前端控件 前端界面展示依旧以from-create ,因此所有的交互都置于后端，我们需要在原有的表单类型中添加一些复杂表单，比如联动效果，事件触发（最好能够将复杂交互在内部实现，插件使用上不要让用户过于繁琐，否则教育成本太高）具体参加：dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/params 任务插件目前同数据库的交互有两块：数据源、资源中心。这里不变，spi 只保存相关的具体信息。查询调用走老逻辑即可，表单中会有相关的选择。 3:扩展接口 扩展接口（用户插件实现需要关注的两个接口，具体代码我还没有提，可能会有修改，但主要是这两块） 所有的任务插件需要实现任务执行的接口（主要是两块：cancel（任务取消）、以及submit（任务执行）） 所有的任务插件需要实现任务UI的相关接口（用于界面展示，大体同alert-plugin类似，api（UiPluginController不需要做什么修改，仅仅需要增加一个类型）） 我们会把插件的相关信息存储到plugin_define表中。 需要注意的是，我们依然要尽可能减少插件依赖。 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
