Mighten opened a new issue, #16482: URL: https://github.com/apache/dolphinscheduler/issues/16482
### Search before asking - [X] I had searched in the [DSIP](https://github.com/apache/dolphinscheduler/issues/14102) and found no similar DSIP. ### Motivation Supporting **user-customized K8s YAML Pod** has the following benefits: - **Flexibility:** Unlike the existing K8s low-code job with limited functionality, YAML Pod provide users with the flexibility to define sophisticated task instances in DolphinScheduler, similar to how custom JSON does in DataX. - **Workflow Customization:** Users can integrate operational and maintenance processes into DolphinScheduler using YAML Pod for complex workflows. In short, by enabling user-customized YAML Pod, *DolphinScheduler* can better support a wide range of Kubernetes-based workflows and operational requirements. ### Design Detail ### Frontend Design The frontend adds support for user-customized k8s YAML Pod while remaining compatible with the original k8s low-code jobs. 1. **The Web UI layouts**  When the user switches on the Custom Template, the Low-code k8s Job fields should *hide* and YAML editor should *appear* (or vice versa), similar to the JSON Custom Template in the *DataX* plugin. This feature is implemented using the *Vue component span*, which is controlled by *reactive variables* (such as `yamlEditorSpan`) in the file `dolphinscheduler-ui/src/views/projects/task/components/node/fields/use-k8s.ts`. 3. **The Request body** When the user switches to Custom Template mode, the request body should include only YAML-related fields (`customConfig` and `yamlContent`), and all previously hidden fields should not be sent. This feature is implemented using the `taskParams` in the file `dolphinscheduler-ui/src/views/projects/task/components/node/format-data.ts` 4. **i18n/locales** Apache DolphinScheduler is an international software and should support multiple languages. The text on the Web UI are retrieved from variables defined in the file `dolphinscheduler-ui/src/locales/{en_US, zh_CN}/project.ts`. And for user-customized k8s YAML tasks, there are three key variables to consider: - `k8s_custom_template`: the label for the switch to enable user-customized k8s YAML tasks. - `k8s_yaml_template`: the label for the text editor used to input user YAML. - `k8s_yaml_empty_tips`: the warning message displayed when a user tries to submit empty YAML This feature is implemented by invoking `t('project.node.${variable_name}')` (such as `t('project.node.k8s_yaml_template')`) in the file `dolphinscheduler-ui/src/views/projects/task/components/node/fields/use-k8s.ts`. ### Backend Design The backend design describes the process of how the worker executes user-customized k8s YAML Pod tasks.  After the worker checks the parameters, `K8sYamlTaskExecutor` is loaded for the current user-customized k8s YAML Pod task. Once the YAML is parsed into `HasMetadata`, its `kind` field is used to assign `abstractK8sOperation` as `K8sPodOperation` for executing the YAML Pod task. 1. **K8s Task Executors**  Three k8s task executor are involved: - `AbstractK8sTaskExecutor` is an abstract class that represents a k8s task executor. - `K8sTaskExecutor` is a concrete class that *extends* `AbstractK8sTaskExecutor` to represent a low-code executor - `K8sYamlTaskExecutor` is a concrete class that *extends* `AbstractK8sTaskExecutor` to represent a user-customized k8s YAML task executor. 2. **K8s Operation handlers**  Two operation handlers are involved: - `AbstractK8sOperation` is an interface representing all k8s resource operations. - `K8sPodOperation` is a concrete class that implements `AbstractK8sOperation` to handle *Pod* operations ### Usecase Design A typical use case for a k8s YAML Pod task includes *uploading* YAML, *online* workflows, and *starting* workflows, similar to k8s low-code jobs, unless users switch to the *Custom Template* option to fill in Pod YAML.  1. The user **edits** a k8s Pod YAML node in a workflow 2. If the *Custom Template* is activated and YAML content is not blank, the user may **online** this whole workflow 3. If the workflow is *online*, the user may **start** the workflow and review the logs generated during the execution of the workflow. ### Compatibility, Deprecation, and Migration Plan The user-customized k8s YAML feature requires only `customConfig` to be activated(1). By default, the value is 0, which applies to the existing k8s low-code jobs. This DSIP does not propose any deprecation or migration plan, but community feedback is valued. ### Test Plan The User-customized k8s YAML Pod feature allows users to submit YAML *Pod* to k8s. And this test plan aims to ensure that the feature functions as expected and meets user requirements. | Test Case #| Name | Action | Expectation | | :--------: | :----------------| :------------------------- | :------------------------------------ | | 1 | UI Display | Edit YAML, save and reopen | The YAML content stays up-to-date. | | 2 | UI Validation | try to submit empty YAML | The UI modal dialog intercepts empty YAML. | | 3 | Online Workflow | Save workflow, and online | The User successfully brings the workflow online. | | 4 | Dryrun Workflow | Run workflow as dryrun mode| The Master successfully dry runs this task. | | 5 | Test Workflow | Run workflow as test mode | The Worker successfully tests this task. | | 6 | Run Workflow | Run workflow | The Worker successfully runs this task. | ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
