?????? [DISCUSS] Process definithon json split design

yongqing Fri, 27 Nov 2020 00:05:33 -0800

+1




------------------&nbsp;????????&nbsp;------------------
??????:                                                                         
                                               "dev"                            
                                                        
<[email protected]&gt;;
????????:&nbsp;2020??11??27??(??????) ????4:01
??????:&nbsp;"[email protected]"<[email protected]&gt;;

????:&nbsp;Re: [DISCUSS] Process definithon json split design



hi:



- The Master is only responsible for DAG scheduling. When a job is
executed, it sends the job code to the Worker, and the Worker 
is&nbsp;&nbsp;&nbsp; 


+ 1




This has a great impact if - 1 is not implemented






--------------------------------------
BoYi ZhangE-mail : [email protected]
On 11/27/2020 15:53??Hemin Wen<[email protected]&gt; wrote??
There are two solutions for the query timing of job details:
- The Master is only responsible for DAG scheduling. When a job is
executed, it sends the job code to the Worker, and the Worker is
responsible for querying the job details and executing the job
- The Master executes DAG scheduling, queries the job details when a job is
executed, and then sends it to the Worker to execute the job

---------------------------------------------------------------------------------------------------------

????????????????????????????????????????

- 
Master??????DAG??????????????????????????????????????Worker??Worker??????????????????????????
- Master????DAG??????????????????????????????????????????????Worker????????

--------------------
DolphinScheduler(Incubator) Commtter
Hemin Wen&nbsp; ??????
[email protected]
--------------------


Hemin Wen <[email protected]&gt; ??2020??11??25?????? ????10:01??????

Hi!

About json splitting of workflow definition, The following is the design
plan for splitting three tables.

Everyone can discuss together.


--------------------------------------------------------------------------------------------------------------

## 1. Currently
The workflow definition of the current DS system includes task definition
data and task relationship data. In the design of the database, task data
and task relationship data are stored in the workflow as a string type
field (process_definition_json) Definition table (t_ds_process_definition).

With the increase of workflow and tasks, the following problems will arise:

-Task data, relational data and workflow data are coupled together, which
is not friendly to the scenario of single-task scheduling. The task must be
created in the workflow

-The task cannot be reused because the task is created in the workflow

-The maintenance cost is high. If you move the whole body and modify any
task, you need to update the data in the workflow as a whole, and it also
increases the log cost

-When there are many tasks in the workflow, the efficiency of global
search and statistical analysis is low, such as querying which tasks use
which data source

-Poor scalability, for example, the realization of blood relationship
function in the future will only lead to more and more bloated workflow
definitions

-Tasks, relationships, and workflow boundaries are blurred. Condition
nodes and delay nodes are also regarded as a task, which is actually a
combination of relationships and conditions

Based on the above pain points, we need to redefine the business
boundaries of tasks, relationships, and workflows, and redesign their data
structures based on this

## 2. Design Ideas

### 2.1 Workflow, relation, job

First of all, we set aside the current implementation and clarify the
business boundaries of tasks (the subsequent description is changed to
jobs), relationships, and workflows, and how to decouple

-Job: the task that the scheduling system really needs to execute, the job
only contains the data and resources needed to execute the job
-relation: the relationship between the job and the job and the execution
conditions, including the execution relationship (after A completes,
execute B) and execution conditions (after A completes and succeeds,
execute B; after A completes and fails, execute C; A completes 30 After
minutes, execute D)
-Workflow: the carrier of a set of relationships, the workflow only saves
the relationships between jobs (DAG is a display form of workflow, a way to
create relationships)

Combined with the functions supported by the current DS, we can make a
classification

-Job: Dependency check, sub-process, Shell, stored procedure, Sql, Spark,
Flink, MR, Python, Http, DataX, Sqoop
-Relationship: serial execution, parallel execution, aggregate execution,
conditional branch, delayed execution
-Workflow: the boundary of scheduling execution, including a set of
relationships

#### 2.1.1 Further refinement

The job definition data is not much different from the current job
definition data. Both are composed of public fields and custom fields. You
only need to remove the fields related to the relationship.

The workflow definition data is not much different from the current
workflow definition data, just remove the json field.

Relational data, we can abstract into two nodes and one path according to
classification. The node is the job, and the path includes the conditional
rules that need to be met from the pre-node to the post-node. The
conditional rules include: unconditional, judgment condition, and delay
condition.

### 2.2 Version Management

We clarify the business boundaries. After decoupling, they become a
reference relationship. The workflow and the relationship are one-to-many,
and the relationship and the job are one-to-many. Not only is the
definition of data, we also need to consider instance data. Every time a
workflow is scheduled and executed, a workflow instance will be generated.
Jobs and workflows can be changed, and the workflow instance must support
viewing, rerun, recovery failure, etc. . This requires the introduction of
version management of the definition data. Every time workflow,
relationship, and job changes need to save old version data and generate
new version data.

So the design idea here is:

To define data, you need to add a version field

The definition table needs to add the corresponding log table

When creating definition data, double write to the definition table and
log table. When modifying the definition data, save the modified version to
the log table

There is no need to save version information in the reference data of the
definition table (refer to the latest version), and the version information
at the time of execution is saved in the instance data

### 2.3 Coding Design

This also involves the import and export of workflow and job definition
data. According to the previous community discussion, a coding scheme needs
to be introduced. Each piece of data in workflow, relationship, and job
will have a unique code. Related Issues: https://github
.com/apache/incubator-dolphinscheduler/issues/3820

Resource: RESOURCE_xxx

Task: TASK_xxx

Relation: RELATION_xxx

Workflow: PROCESS_xxx

Project: PROJECT_xxx

## 3. Design plan

### 3.1 Table model design

#### 3.1.1 Job definition table: t_ds_task_definithon

| Column Name | Description |
| ----------------------- | -------------- |
| id | Self-incrementing ID |
| union_code | unique code |
| version | Version |
| name | Job name |
| description | description |
| task_type | Job type |
| task_params | Job custom parameters |
| run_flag | Run flag |
| task_priority | Job priority |
| worker_group | worker group |
| fail_retry_times | Number of failed retries |
| fail_retry_interval | Failure retry interval |
| timeout_flag | Timeout flag |
| timeout_notify_strategy | Timeout notification strategy |
| timeout_duration | Timeout duration |
| create_time | Creation time |
| update_time | Modification time |

#### 3.1.2 Task relation table: t_ds_task_relation

| Column Name | Description |
| ----------------------- | ------------------------- ------------- |
| id | Self-incrementing ID |
| union_code | unique code |
| version | Version |
| process_definition_code | Workflow coding |
| node_code | Node code (workflow code/job code) |
| post_node_code | Post node code (workflow code/job code) |
| condition_type | Condition type 0: None 1: Judgment condition 2: Delay
condition |
| condition_params | Condition parameters |
| create_time | Creation time |
| update_time | Modification time |

#### 3.1.3 Workflow definition table: t_ds_process_definithon

| Column Name | Description |
| ---- | ---- |
| id | Self-incrementing ID |
| union_code | unique code |
| version | Version |
| name | Workflow name |
| project_code | Project code |
| release_state | Release state |
| user_id | Owning user ID |
| description | description |
| global_params | Global parameters |
| flag | Whether the process is available: 0 is not available, 1 is
available |
| receivers | recipients |
| receivers_cc | CC |
| timeout | Timeout time |
| tenant_id | tenant ID |
| create_time | Creation time |
| update_time | Modification time |

#### 3.1.4 Job definition log table: t_ds_task_definithon_log

Add operation type (add, modify, delete), operator, and operation time
based on the job definition table

#### 3.1.5 Job relation log table: t_ds_task_relation_log

Add operation type (add, modify, delete), operator, and operation time
based on the job relationship table

#### 3.1.6 Workflow definition log table: t_ds_process_definithon_log

Add operation type (add, modify, delete), operator, and operation time
based on the workflow definition table

### 3.2 Frontend

*The design here is just a personal idea, and the front-end help is needed
to design the interaction*

Need to add job management related functions, including: job list, job
creation, update, delete, view details operations

To create a workflow page, you need to split json into workflow definition
data and job relationship data to the back-end API layer to save/update

Workflow page, when dragging task nodes, add reference job options

The conditional branch nodes and delay nodes need to be resolved into the
conditional rule data in the relationship; conversely, the conditional rule
data returned by the backend needs to be displayed as the corresponding
node when querying the workflow

### 3.3 Master

When the Master schedules the workflow, you need to modify <Build dag from
json&gt; to <Build dag from relational data&gt;. When executing a workflow, 
first
load the relational data in full (no job data is loaded here), generate
DAG, and traverse DAG execution , And then get the job data that needs to
be executed

Other execution processes are consistent with existing processes


--------------------------------------------------------------------------------------------------------------

## 1.????

????DS??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????process_definition_json??????????????????????????????t_ds_process_definition??????

????????????????????????????????????????

- 
??????????????????????????????????????????????????????????????????????????????????????????

- ????????????????????????????????????????

- 
????????????????????????????????????????????????????????????????????????????????????????????

- ??????????????????????????????????????????????????????????????????????????????

- ??????????????????????????????????????????????????????????????

- 
??????????????????????????????????????????????????????????????????????????????????????

????????????????????????????????????????????????????????????????????????????????????????

## 2.????????

### 2.1 ??????????????????

????????????????????????????????????????????????????????????????????????????????????????????

- ??????????????????????????????????????????????????????????????????
-
??????????????????????????????????????????????????????A????????????B??????????????A??????????????????B??A??????????????????C??A????30????????????D??
- 
??????????????????????????????????????????????????DAG????????????????????????????????????????????

????????DS??????????????????????????????

- 
????????????????????????Shell????????????Sql??Spark??Flink??MR??Python??Http??DataX??Sqoop
- ??????????????????????????????????????????????????????
- ????????????????????????????????????

#### 2.1.1 ??????????

??????????????????????????????????????????????????????????????????????????????????????????????????????????????

??????????????????????????????????????????????????????json??????????????


????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

### 2.2 ????????


??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

??????????????????????

????????????????????????

??????????????????????????

??????????????????????????????????????????????????????????????????????????????

????????????????????????????????????????????????????????????????????????????????????

### 2.3 ????????

??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????Issue??
https://github.com/apache/incubator-dolphinscheduler/issues/3820

??????RESOURCE_xxx

??????TASK_xxx

??????RELATION_xxx

????????PROCESS_xxx

??????PROJECT_xxx

## 3.????????

### 3.1 ??????????

#### 3.1.1 ????????????t_ds_task_definithon

| 
????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | ????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
| ----------------------- | -------------- |
| 
id&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | ????ID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
| 
union_code&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | ????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
| 
version&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | ????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
| 
name&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | ????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
| 
description&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | ????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
| 
task_type&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | ????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
| 
task_params&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | ?????????????? |
| 
run_flag&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | ????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
| task_priority&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 
??????????&nbsp;&nbsp;&nbsp;&nbsp; |
| 
worker_group&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
| worker????&nbsp;&nbsp;&nbsp;&nbsp; |
| fail_retry_times&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 
????????????&nbsp;&nbsp; |
| fail_retry_interval&nbsp;&nbsp;&nbsp;&nbsp; | ????????????&nbsp;&nbsp; |
| 
timeout_flag&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
| ????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
| timeout_notify_strategy | ????????????&nbsp;&nbsp; |
| timeout_duration&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 
????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
| 
create_time&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | ????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
| 
update_time&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | ????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |

#### 3.1.2 ????????????t_ds_task_relation

| 
????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | 
????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| ----------------------- | -------------------------------------- |
| 
id&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | 
????ID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| 
union_code&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | 
????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| 
version&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | 
????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| process_definition_code | 
??????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| 
node_code&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | ????????????????????/??????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
| post_node_code&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 
????????????????????????/??????????&nbsp;&nbsp;&nbsp; |
| condition_type&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 
???????? 0???? 1?????????? 2?????????? |
| condition_params&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 
????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| 
create_time&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | 
????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| 
update_time&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 | 
????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |

#### 3.1.3 ??????????????t_ds_process_definithon

| ???? | ???? |
| ---- | ---- |
| id&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 
????ID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| union_code&nbsp;&nbsp;&nbsp; | 
????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| version&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 
????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| name&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 
??????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| project_code&nbsp; | 
????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| release_state | 
????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| user_id&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 
????????ID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| description&nbsp;&nbsp; | 
????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| global_params | 
????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| flag&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | ??????????????0 
????????1 ???? |
| receivers&nbsp;&nbsp;&nbsp;&nbsp; | 
??????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| receivers_cc&nbsp; | 
??????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| timeout&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 
????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| tenant_id&nbsp;&nbsp;&nbsp;&nbsp; | 
????ID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| create_time&nbsp;&nbsp; | 
????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |
| update_time&nbsp;&nbsp; | 
????????&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 |

#### 3.1.4 ????????????????t_ds_task_definithon_log

??????????????????????????????????????????????????????????????????

#### 3.1.5 ????????????????t_ds_task_relation_log

??????????????????????????????????????????????????????????????????

#### 3.1.6 ??????????????????t_ds_process_definithon_log

????????????????????????????????????????????????????????????????????

### 3.2 ????

*??????????????????????????????????????????????????*

??????????????????????????????????????????????????????????????????????????????

??????????????????????json??????????????????????????????????????????API??????/????

????????????????????????????????????????????

????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

### 3.3 Master


Master????????????????????<??json????dag&gt;??????<??????????????dag&gt;????????????????????????????????????????????????????????????????DAG??????DAG????????????????????????????????

??????????????????????????

--------------------
DolphinScheduler(Incubator) Commtter
Hemin Wen&nbsp; ??????
[email protected]
--------------------

?????? [DISCUSS] Process definithon json split design

Reply via email to