SEPURI-SAI-KRISHNA opened a new issue, #18543: URL: https://github.com/apache/dolphinscheduler/issues/18543
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues. ### What happened `TaskInstanceMapper#findLastTaskInstances` is supposed to return **the last task instance per task code** of a workflow instance. It resolves "last" by joining on the maximum `end_time`: ```xml from t_ds_task_instance instance join ( select task_code, max(end_time) as max_end_time, workflow_instance_id from t_ds_task_instance where 1=1 and workflow_instance_id = #{workflowInstanceId} and state != 8 ... group by task_code ) t_max on instance.workflow_instance_id = t_max.workflow_instance_id and instance.task_code = t_max.task_code and instance.end_time = t_max.max_end_time ``` `end_time` is not unique. When two attempts of the same task share the very same `end_time`, both of them match `instance.end_time = t_max.max_end_time` and the query returns **two rows for one task code**. This is easy to hit on MySQL, where `t_ds_task_instance.end_time` is declared as ```sql `end_time` datetime DEFAULT NULL COMMENT 'task end time', ``` a `datetime` **without fractional seconds**, so every timestamp is truncated to a whole second. A task that fails and is retried quickly (a short or zero failed-retry interval, or simply a fast-failing task) ends up with the failed attempt and the retry sharing an identical `end_time`. Note the query filters on `state != 8` only — it does not filter on `flag`, so the invalidated previous attempts are included as well. The only caller is `DependentExecute#dependResultByAllTaskOfWorkflowInstance`, which keys the result by task code: ```java Map<Long, TaskExecutionStatus> taskExecutionStatusMap = taskInstanceList.stream() .filter(taskInstance -> taskInstance.getTaskExecuteType() != TaskExecuteType.STREAM) .collect(Collectors.toMap(TaskInstance::getTaskCode, TaskInstance::getState)); ``` `Collectors.toMap` has no merge function, so a duplicated task code throws ``` java.lang.IllegalStateException: Duplicate key <taskCode> (attempted merging values ... and ...) ``` and the dependency evaluation of the DEPENDENT task blows up. ### What you expected to happen `findLastTaskInstances` returns at most one task instance per task code — the last attempt — so a DEPENDENT task can evaluate the upstream workflow normally. ### How to reproduce 1. Workflow **A** contains a task that fails and then succeeds on retry, with a short failed-retry interval so that both attempts finish within the same second (on MySQL any two attempts finishing in the same second are enough). 2. Workflow **B** contains a DEPENDENT task depending on workflow **A** with **"ALL tasks"** selected (`DEPENDENT_ALL_TASK_CODE`). 3. Run A to completion, then run B. 4. B's dependent check fails with `IllegalStateException: Duplicate key` instead of resolving the dependency. It also reproduces directly at the DAO level — insert two task instances with the same `task_code` and the same `end_time` into one workflow instance and call `queryLastTaskInstanceListIntervalInWorkflowInstance`; it returns 2 rows instead of 1. ### Anything else The same duplication would silently affect any future caller of this query, since the method name (`findLastTaskInstances` / `queryLastTaskInstanceListIntervalInWorkflowInstance`) promises one row per task code. ### Version dev ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
