legen618 commented on pull request #3576: URL: https://github.com/apache/incubator-dolphinscheduler/pull/3576#issuecomment-688675611
Because the number of commit is relatively large and messy, which may affect the efficiency of code review, I hereby explain each modified file: **For the function that changes the task state to force success:** Mainly adds an interface. For a single failed task specified by the user, the task state can be changed to force success (FORCED_SUCCESS). The following files have been modified: - ` TaskInstanceController. Java ` : increase the API - 'TaskInstanceService. Java' : The logical implementation of the above API, the general process is to determine whether the user has operation permission, verify whether the specified task instance exists, determine whether the state of the task instance is **typeIsFailure**, if these conditions are met, then modify the state of the task instance in the database - ` TaskInstanceControllerTest. Java ` : API corresponding unit test - ` TaskInstanceServiceTest. Java ` : logic to implement the corresponding unit tests - 'executionstatus. Java' : Add the status FORCED_SUCCESS to the decision for the TypeIsSuccess function **For the running workflow to verify whether there is a mandatory success:** is mainly to modify 'masterexecthread.java', in the 'runProcess()' method to continuously scan the current workflow to see whether any failed tasks have been successfully enforced, that is, to traverse the state of errorList in the database; Remove from the errorList, if any, and submit the task after the node. - one other note here is that I have included a change in pull request#3375 to fix a bug that determines if the task dependency is satisfied. **Running from force success for stopped workflows:** this is mainly the addition of a new commandType that operates on the workflows using the original Execute interface. The process of execution reads in the entire DAG, then checks from start to finish. Tasks marked FORCED_SUCCESS that were not performed before are treated as' typeIsSuccess '. Modified the following file: - 'executetype.java' : Add a new type called **RESUME_FROM_FORCED_SUCCESS** - 'ExecutorService.java' : Adds some validation of the new executeType, the workflow is stopped to perform the restore, and all task instances with flag 1 on the last run must have been enforced to succeed. - 'ExecutorServiceTest2' : Corresponding unit test - 'commandType. Java' : adds **RESUME_FROM_FORCED_SUCCESS** - 'processService.java' : Added handling for **RESUME_FROM_FORCED_SUCCESS** when building workflow. Contains a few small functions that recursively determine whether a task instance has been forced to succeed in a nested sub-process. - 'taskinstancemapper. Java', 'taskinstancemapper. XML' : the interface and SQL statement corresponding to the database access is the dependency of the new content in ProcessService. Java. - 'daghel.java' : This is mainly a modification of getStartVertex because the recovery process from the forced success will load the complete DAG, and the judgment process will require some additional judgment logic **For some Status codes:** uniformly added in 'status.java' In addition to the above, the four corresponding files have been modified as follows because **TaskType** and **CommandType** have been added: - `AlertManager.java` - `BusinessTimeUtils.java` *The above should be easy to understand* - ` DataAnalysisServiceImpl. Java ` and ` TaskCountDto. Java ` : the two is a modification of various task state statistics, increased FORCED_SUCCESS this state. ----- 因为我的commit数量比较多而且相对比较凌乱，可能影响code review的效率，所以在这里对每一个改动的文件加以说明： **对于修改任务状态为强制成功这一功能：**主要是增加一个接口，对用户指定的单个失败任务，可以任务状态修改为强制成功过（FORCED_SUCCESS）。修改了以下几个文件： - `TaskInstanceController.java`: 增加api - `TaskInstanceService.java`: 上述api对应的逻辑实现，大概流程是判断用户是否有操作权限，检验指定的任务实例是否存在，判断任务实例的状态是否为**typeIsFailure**，如果这些条件都满足了则修改数据库中任务实例的状态 - `TaskInstanceControllerTest.java`: api对应的单元测试 - `TaskInstanceServiceTest.java`: 逻辑实现对应的单元测试 - `ExecutionStatus.java`: 增加FORCED_SUCCESS这种状态，同时将其加入TypeIsSuccess函数的判断中 **对于运行的工作流检验是否有强制成功的发生：**主要是修改了`MasterExecThread.java`，在`runProcess()`方法中持续扫描当前工作流中是否有失败的任务被强制成功了，即遍历errorList在数据库中的状态；如果有的话就从errorList中移除，同时submit节点之后的任务。 - 这里还需要说明一点，就是我把pull request#3375的改动也加进来了，就是修复一个判断任务依赖是否满足的bug。 **对于停止的工作流从强制成功处运行**：主要是增加一个新的commandType，利用原来的execute接口对工作流来进行操作。执行的过程会读入整个dag，然后从头到尾的检测，执行之前没有执行的，标记为FORCED_SUCCESS的任务会当做`typeIsSuccess`。修改了以下的文件： - `ExecuteType.java`: 增加一种新的类型，叫 **RESUME_FROM_FORCED_SUCCESS** - `ExecutorService.java`: 增加了对新executeType的一些检验，工作流处于停止状态才能执行恢复的操作，同时上一次运行中对应的所有flag为1的任务实例中必须要有被强制成功过的。 - `ExecutorServiceTest2`: 对应的单元测试 - `CommandType.java`: 增加 **RESUME_FROM_FORCED_SUCCESS** - `ProcessService.java`: 增加构建工作流时对 **RESUME_FROM_FORCED_SUCCESS** 的处理。包含了一些小的函数，用来递归的判断嵌套的sub-process中是否有强制成功过的任务实例。 - `TaskInstanceMapper.java`，`TaskInstanceMapper.xml`: 对应的数据库访问的接口和SQL语句，就是ProcessService.java的中新增内容的依赖。 - `DagHelper.java`: 主要是修改了getStartVertex这个函数，因为从强制成功处恢复的过程会载入完整的dag，判断的过程需要增加一些额外的判断逻辑 **对于一些状态码：**统一增加在`Status.java`中 除上述而外，因为新增了**TaskType**和**CommandType**，所以修改了4个对应的文件如下： - `AlertManager.java` - `BusinessTimeUtils.java` *以上应该很容易理解* - `DataAnalysisServiceImpl.java`和`TaskCountDto.java`: 这两个是对各种任务状态统计的一个修改，增加了FORCED_SUCCESS这个状态。 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
