Regarding plug-inization, the front-end can use form-create, which supports json format data, but the UI only supports iview and element-ui, and the existing ans-ui needs to be replaced. The form-create that personally feels is better compatible with the page style, and the document details Wait, form-create link address: http://form-create.com/
DolphinScheduler(Incubator) committer Caibiao Xiang ?????? [email protected] ------------------ ???????? ------------------ ??????: "JUN GAO"<[email protected]>; ????????: 2020??6??24??(??????) ????4:33 ??????: "dev"<[email protected]>; ????: [Proposal] Alert Plugin Design Hi All, This is a part of the [Proposal] DolphinScheduler Plugin Design https://github.com/apache/incubator-dolphinscheduler/issues/2869 Welcome all partners who are interested in this issue to participate in the discussion and development contribution Summary designPlugin definition & plugin instances The core of the plug-in design is that the upper layer logic and the specific implementation are decoupled by SPI. When the system uses the plug-in, it should create a plug-in instance based on the plug-in supported by the current system. For example, the mail alert plug-in will provide the required parameter list. When used in the system, an instance of the plug-in needs to be created based on the parameter list. Another example is the Job of MR task, the Job plug-in will define the required parameters, and then in the DAG we create the Job and generate an instance of the Job plug-in. Configuration parameters & configuration files In addition, there are some problems in configuring the alert through the alert.properties method. For example, in some enterprises, multiple departments work on the DS, and different departments need to configure different WeChat alert and e-mail alert stmp services. This scenario cannot be met by alert.properties. Therefore, we need to support the function of creating multiple alert mode instances under one alert mode plug-in. In order to instantiate this capability and reduce the configuration files to simplify installation and deployment, I provide all the parameters required by the plug-in to the front end as a parameter template for the plug-in, fill in when creating an instance of the plug-in on the front-end page, you can achieve more An email alert function, and no longer need to modify the alert.properties file through the background. alert group & DS user In the current DS version, the alert group is a collection of users. The alert service finds the user list through the alert group when sending an alert, and then obtains the email address of each user and adds it to the recipient of the email alert. However, we found that we can't manage WeChat alert and DingTalk alert through alert groups. WeChat and nail alert can only be configured and used through the backend alert.properties configuration file. All we intend to redefine the alert group, the alert group is defined as a collection of alert plug-in instances, rather than just a collection of users. workflow define & alert group There are two fields in the current process schedule, e-mail recipient and cc sender, which binds the process definition to e-mail alerts to a certain extent. The actual situation is that when the process is defined, the alert may not be used, or an alert method other than email may be used. So we need to redefine the relationship between the process definition and the alert and alert group. Front-end UI plug-in http://www.alpacajs.org/docs/api/forms.html The front-end architecture supports the generation of front-end UI components based on json. In SPI, I encapsulated the parameters of the plug-in, and all the plug-in parameters will be automatically converted into json supported by alpacajs. User path Based on the above design considerations, we have redefined the user path of the alert function to make it more scalable and easier to understand and use. The modified user path is as follows: If the user needs to use the alert function, the user can choose to create an alert instance based on the existing alert plug-in in the alert group creation and management page. For example, the user creates an alert group AlertGroupA, and then on the management page of AlertGroupA, you can choose to create a specific alert method through the "Add Alert Method" operation. When "Add Alert Method" is displayed, the existing alert plug-in (plug-in The information can be put into plugin_define in the plug-in definition table in db). The user selects the corresponding plug-in, and opens the creation page, which displays all the required parameters of the alert plug-in. After the user completes, an alert method instance is created. Multiple instances of multiple alert methods can be created in an alert group. The process definition is associated with an alert group ID. The process definition is no longer associated with a specific alert method, but is instead associated with an alert group. In this way, even if you need to modify the alert method in the future, you only need to modify the alert method instance in the alert group management, and no longer need to modify the process definition. detailed designDatabase table designPlugin definition table plugin_define id, plugin_name, plugin_params Plugin instance table plugin_instance id plugin_define_id, plugin_instance_params Interface designincubator-dolphinscheduler-maven-plugin https://github.com/gaojun2048/incubator-dolphinscheduler-maven-plugin A custom maven plug-in, its role is to automatically check the spi and plug-in related dependencies, and can support adding dolphinscheduler-plugin logo in the pom file to allow DS to automatically generate related META-INF/services files at compile time. For detailed information, please refer to: https://github.com/gaojun2048/incubator-dolphinscheduler-maven-plugin SPIDolphinSchedulerPlugin DS plug-in top-level interface, all DS plug-ins must implement this interface AlertChannelFactory Alert plug-in factory interface. All alert plug-ins need to implement this interface. This interface is used to define the name of the alert plug-in and the required parameters. The create method is used to create a specific alert plug-in instance. AlertChannel The interface of the alert plug-in. The alert plug-in needs to implement this interface. There is only one method process in the interface. The upper alert system will use this method and obtain the return information of the alert through the AlertResult returned by the method. AlertData alert content information, including id, title, content, log. AlertInfo For alert-related information, when an upper-layer system calls an alert plug-in instance, the instance of this type is passed into the specific alert plug-in through the process method. It contains the alert content AlertData and the parameter information filled in by the front end of the alert plug-in instance called. AlertResult The alert plugin sends alert return information. org.apache.dolphinscheduler.spi.params Under this package is the plug-in parameter definition. Our front-end uses the front-end library alpacajs http://www.alpacajs.org/docs/api/forms.html, which can dynamically generate the front-end based on the parameter list json returned by the plug-in definition Ui. Under this package, we currently only encapsulate RadioParam, TextParam, and PasswordParam, which are used to define text type parameters, radio parameters, and password type parameters, respectively. AbsPluginParams This class is the base class for all parameters, and these classes inherit from RadioParam. Each DS alert plugin will return a list of AbsPluginParams in the implementation of AlertChannelFactory. org.apache.dolphinscheduler.alert.plugin Under this package is the relevant code for the DS Alert module to load the Alert plug-in. In the AlertChannelManager class, we also need to save the loaded plugin to the database's plugin_define table. some code of this issue gaojun2048#1 <https://github.com/gaojun2048/incubator-dolphinscheduler/pull/1> todo list 1. Improvement of the alert group function, in the alert group, an alert instance can be created based on the existing alert plug-in. 2. Delete recipients and CCs in the process definition and select the associated alert group instead. 3. Submit the dolphinscheduler-maven-plugin to the community. 4. Pull a separate branch from dev for development. 5. Transformation of WeChat alert function. 6. Renovation of the nail warning function. 7. Improvement of mail alert function. 8. Upgrade related tool development from the old version. ?????????????????????????? ??????????????????????????????????????SPI????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????MR??????Job????Job??????????????????????????????DAG????????????Job????????????Job???????????? ????????&???????? ????????????alert.properties????????????????????????????????????????????????????????DS??????????????????????????????????????????????????stmp????????????????alert.properties????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????alert.properties?????? ????????DS?????????? ????????DS????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????alert.properties???????????????????????????????????????????????????????????????????????????????????????????????????????? ???????????????????? ?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? ????UI?????? http://www.alpacajs.org/docs/api/forms.html ??????????????????json????????ui??????SPI????????????????????????????????????????????????????????alpacajs??????json?? ???????? ???????????????????????????????????????????????????????????????????????????????????????????????????? ?????????????????????? ????????????????????????????????????????????????????????????????????????????????????????????????????????????????????AlertGroupA????????AlertGroupA????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????db????????????????plugin_define???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? ???????????????????????????????????????????? ??????????????????ID?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? ?????????????????????????????? plugin_define id, plugin_name, plugin_params ?????????? plugin_instance id plugin_define_id, plugin_instance_params ????????incubator-dolphinscheduler-maven-plugin https://github.com/gaojun2048/incubator-dolphinscheduler-maven-plugin ??????????maven????????????????????????????spi????????????????????????????????pom??????????dolphinscheduler-plugin??????????DS??????????????????????META-INF/services???????????????????????? ?? https://github.com/gaojun2048/incubator-dolphinscheduler-maven-plugin SPIDolphinSchedulerPlugin DS??????????????????DS?????????????????????? AlertChannelFactory ????????????????????????????????????????????????????????????????????????????????????????create??????????????????????????????. AlertChannel ????????????????????????????????????????????????????????????process ????????????????????????????????????????????AlertResult?????????????????????? AlertData ??????????????????id???????????????????? AlertInfo ??????????????????????????????????????????????????????????process????????????????????????????????????????AlertData?????????????????????????????????????????? AlertResult ?????????????????????????? org.apache.dolphinscheduler.spi.params ??????????????????????????????????????alpacajs?????????? http://www.alpacajs.org/docs/api/forms.html????????????????????????????????????json????????????????ui?? ??package??????????????????RadioParam??TextParam??PasswordParam??????????????text????????????radio?????? password???????????? AbsPluginParams ??????????????????????RadioParam????????????????????????DS????????????????AlertChannelFactory????????????????AbsPluginParams??list?? org.apache.dolphinscheduler.alert.plugin ????????DS Alert????????Alert????????????????????????????AlertChannelManager??????????????????????????????????????????plugin_define?????? ??????Issue?????????? gaojun2048#1 <https://github.com/gaojun2048/incubator-dolphinscheduler/pull/1> todo list 1. ?????????????????????????????????????????????????????????????? 2. ?????????????????????????????????????????????????? 3. ??dolphinscheduler-maven-plugin???????????? 4. ??dev?????????????????????? 5. ?????????????????? 6. ???????????????????? 7. ???????????????????? 8. ???????????????????????????? ??????????issue?????????????????????????????????? -- DolphinScheduler(Incubator) PPMC Jun Gao ???? [email protected]
