??????[DISCUSS] Create workflow, front end and back end interaction optimization(Json split project)

[Jave-Chen]

Hi, Wen
In a concurrent scenario, if two or more people edit a 
workflow dag at the same time, will there be a problem 
of locking the database table?




------------------ ???????? ------------------
??????:                                                                         
                                               "dev"                            
                                                        
<wenhe...@apache.org&gt;;
????????:&nbsp;2021??7??1??(??????) ????5:44
??????:&nbsp;"dev"<dev@dolphinscheduler.apache.org&gt;;

????:&nbsp;[DISCUSS] Create workflow, front end and back end interaction 
optimization(Json split project)



Hi!

Json split design reference??
https://lists.apache.org/thread.html/r84ed046de63899476f067dd207496428812a8687045bb4485072ee2d%40%3Cdev.dolphinscheduler.apache.org%3E

For the current interaction design problems, redesign, if you have
different opinions, you can participate in the discussion.
If support, please +1.

Design objective:

&nbsp;&nbsp;&nbsp; - Remove the processdefinitionjson field in the API when 
creating /
editing a workflow

&nbsp;&nbsp;&nbsp; - When viewing the workflow, the back end does not need to 
return the
task detail data, just return the task relation data

&nbsp;&nbsp;&nbsp; - When editing a workflow, it supports referencing tasks 
that already
exist in the database

&nbsp;&nbsp;&nbsp; - When creating / editing a workflow, the front end only 
needs to send
the necessary task information. For the referenced task, it does not need
to send the task detail data, which reduces the interaction data and
improves the interaction efficiency

Current design:

&nbsp;&nbsp;&nbsp; - New / edit / copy task nodes need to call back-end task 
interface to
maintain a task data

&nbsp;&nbsp;&nbsp; - After building DAG, users click Save workflow. At this 
time, the
front end does not need to send task details data, just need to send task
code

The problems in the current design

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; When editing DAG, the front end 
supports right-click copy of a node
to generate a new node. According to the current design, you need to call
the back-end interface to create a task. That is to say, the newly added
nodes in the workflow creation process will generate a task data in the
database (the workflow details page is not allowed to delete the database
task, because there may be potential dependencies). It is inevitable that
some of the nodes are temporary nodes or newly added and deleted nodes. In
the long run, task garbage data will become more and more difficult to
maintain.

Design optimization:

&nbsp;&nbsp;&nbsp; - The back end opens the function of generating unique 
encoding as an
interface

&nbsp;&nbsp;&nbsp; - When editing DAG and adding / copying nodes, the front end 
calls the
unique encoding interface to generate task encoding, and the version is
empty

&nbsp;&nbsp;&nbsp; - When you edit DAG and reference task, you can directly get 
the code
and version of task

&nbsp;&nbsp;&nbsp; - When the workflow is saved, only the code and version of 
the
referenced task will be sent, and all the task information will be sent to
the new task

&nbsp;&nbsp;&nbsp; - For node relation data, coordinate data and other fields, 
the
front-end does not need to generate task unique ID according to the "task
random number" rule, and all of them are replaced by task code

&nbsp;&nbsp;&nbsp; - When querying the workflow details, the back end does not 
need to
return the task details. When double clicking the task node, the back end
interface is called to query the task details

----------------------------------------------------------------------------------------------------------------------

??????????

- ????????/??????????????API????processDefinitionJson????
- ????????????????????????????task??????????????task????????????
- ??????????????????????????????????????task
- 
????/????????????????????????????????task????????????????task????????????task????????????????????????????????????

??????????

- ????/????/???? task??????????????????task??????????????task????
- 
????????????Dag??????????????????????????????????????task????????????????????task????????

????????????????????

??????Dag??????????????????????????????????????????????????????????????????????????????????????????????task??????????????????????????????????????????????????????????task????????????????????????????????????task??????????????????????????????????????????????????????????????????????????????????????task??????????????????????????????????

??????????

- ????????????????????????????????
- ????Dag??????/????????????????????????????????????task??????????????
- ????Dag??????task????????????????task????????????
- ????????????????????task??????task????????????????task????????task????
- 
??????????????????????????????????????????????????"task-??????"????????task????ID????????????task????
- 
????????????????????????????????task??????????????task????????????????????????task????????

--------------------
Apache DolphinScheduler Commtter
Hemin Wen&nbsp; ??????
wenhe...@apache.org
--------------------