hi developers， 8 days have passed. we have reached a conclusion that Schema 3 will be our final solution for the project/workflow/task.
thanks everybody Best Regards --------------- DolphinScheduler(Incubator) PPMC Lidong Dai 代立冬 [email protected] --------------- On Wed, Dec 9, 2020 at 7:18 PM 3656562 <[email protected]> wrote: > Schema 3 + 1 > > > 原始邮件 > 发件人:leon [email protected] > 收件人:[email protected] > 发送时间:2020年12月7日(周一) 23:56 > 主题:[DISCUSS]Code solution for project/workflow/task > > > ##1. Background *It is suggested to take a look at the design scheme of > splitting workflow JSON* the mail link about json splitting: > https://lists.apache.org/thread.html/r84ed046de63899476f067dd207496428812a8687045bb4485072ee2d%40%3Cdev.dolphinscheduler.apache.org%3E > The current business coding is proposed to be designed on the basis of > splitting workflow JSON, with three purposes: -The references among > workflow, relationship and job, and the external reference of dependent job > and sub workflow job are described by project code, workflow code and > business code -It supports the version management of workflow, job and > resource, and provides a stable business code -Solve the problems of > workflow, job copy, migration, export and import scene ID auto increment, > conflict and so on ##2. Coding design ###2.1 scheme I Project table, > workflow table and job table introduce a new business code field User > manual input / automatically generated by the system. Here, the UI prompts > the user to input the code. Internal reference is established through the > code. Subsequent modification costs are high Project code is globally > unique; workflow code and job code are project level unique Because the > possibility of business code modification is very low, all codes in the > current design do not support modification. The purpose of code > modification can be achieved through the function of export and import When > the workflow and job are copied, the user needs to input the new business > code, and the system will check the conflict When the workflow and job are > exported and imported, the business code (which can be edited) is displayed > on the UI, and the user can choose whether to change the business code; > when exporting, the UI adds the option of whether to export dependent jobs > at the same time When the workflow and job are migrated, the business code > (which can be edited) is displayed on the UI, and the user can choose > whether to change the business code; the project code can be directly > updated after the migration ###2.2 scheme 2 On the basis of scheme 1, there > will be three description words: code, name, description. Code and name are > likely to be repeatedly entered in practical application, while description > is rarely used. Therefore, the current scheme is to replace code with name > without adding a code field. As for the input of English name or Chinese > name, it is up to the user to decide whether to input English name or > Chinese name, and whether to use description for specific function > description field ###2.3 scheme 3 Project code, workflow code and job code > are globally unique. They are only responsible for the uniqueness of > identification data and are not related to business. They are automatically > generated by the system and do not provide page modification methods. 1. > Workflow, job create new workflow (task) / copy workflow from template: > generate new definition_ code/task_ code 2. Export: export exports all the > contents of the current project / workflow / task as a JSON file 3. Import: > check the definition_code/task_code exist -If it is a duplicate of > definition_code/task_code in another project , you will be prompted and the > import failed. -If it is the same as the definition_code/task_code in this > project , you will be prompted to override -If there is no duplicate, all > codes remain unchanged ##Design relation 3 The project code field is > maintained in the workflow table to identify which project the workflow > belongs to The relation table maintains the item code and workflow code > fields, and identifies the workflow of which project the relationship > belongs The relation table maintains the project code and job code of the > reference node (supports jobs that reference other projects) The item code > is maintained in the operation table to identify which project the job > belongs to The referenced workflow / job is determined by project code, > workflow code and job code The referenced workflow is determined by item > code and workflow code in sub workflow-- > ==================================================================== ## > 1.背景 *建议先看一下拆分工作流json的设计方案* 关联邮件地址： > https://lists.apache.org/thread.html/r84ed046de63899476f067dd207496428812a8687045bb4485072ee2d%40%3Cdev.dolphinscheduler.apache.org%3E > 当前业务编码是建议在拆分工作流json基础上设计的，目的有3个： - > 工作流、关系、作业之间的引用，以及依赖作业、子工作流作业对外部的引用通过项目编码、工作流编码、业务编码进行描述 - > 支持工作流、作业、资源的版本管理，提供一个稳定不变的业务编码 - 解决工作流、作业的复制、迁移、导出导入场景的ID自增、冲突等问题 ## 2.编码设计 > ### 2.1 方案一 项目表、工作流表、作业表引入一个新的业务编码字段 > 用户手工输入/系统自动生成，这里UI提示建议用户输入编码，内部通过编码建立引用，后续修改成本高 项目编码全局唯一；工作流编码、作业编码项目级唯一 > 因为业务编码修改的可能性非常低，所以当前设计所有编码都不支持修改，可以通过导出导入功能达到修改编码的目的 > 工作流、作业复制时，需要用户输入新业务编码，系统做冲突检查提醒 > 工作流、作业导出导入时，UI上显示业务编码（可编辑），用户可以选择是否变更业务编码；导出时UI增加是否同时导出依赖作业选项 > 工作流、作业迁移时，UI上显示业务编码（可编辑），用户可以选择是否变更业务编码；迁移直接更新项目编码就可以了 ### 2.2 方案二 > 在方案一基础上，会有三个描述字：code、name、description，code和name在实际应用中很可能是重复输入的，而description又很少应用，所以当前方案是用name替换code，不需要新增code字段，至于输入英文name还是中文name由用户自己决定，段具体的功能描述还是用description字段 > ### 2.3 方案三 项目编码、工作流编码、作业编码全局唯一，只负责标识数据的唯一，不和业务相关，由系统自动生成，不提供页面修改方式。 1. > 工作流、作业从模板创建新工作流（任务）/复制工作流: 生成新的definition_code/task_code 2. 导出： > 导出将当前项目/工作流/任务所有内容导出为json文件 3. 导入： 检查definition_code/task_code是否存在 - > 如果和别的项目的definition_code/task_code重复，则提示并导入失败。 - > 如果和本项目内definition_code/task_code重复，则提示覆盖 - 如果没有重复的，则所有code不变导入 ## 3.引用关系设计 > 工作流表维护项目编码字段，标识工作流归属于哪个项目 关系表维护项目编码、工作流编码字段，标识关系归属于哪个项目的工作流 > 关系表维护引用节点的项目编码、作业编码（支持引用其他项目的作业） 作业表维护项目编码，标识作业归属于哪个项目 > 依赖作业内部通过项目编码、工作流编码、作业编码确定引用的工作流/作业 子工作流内部通过项目编码、工作流编码确定引用的工作流 > DolphinScheduler(Incubator) PPMC BaoLiang 鲍亮 [email protected]
