SEPURI-SAI-KRISHNA opened a new pull request, #18544:
URL: https://github.com/apache/dolphinscheduler/pull/18544
<!--Thanks very much for contributing to Apache DolphinScheduler, we are
happy that you want to help us improve DolphinScheduler! -->
## Was this PR generated or assisted by AI?
YES. The bug was found, and the fix and the test were written with AI
assistance
(Claude Code). I reviewed the change at a high level and verified it via the
added test
and the module test suite.
## Purpose of the pull request
Closes #18543.
`findLastTaskInstances` resolves "the last task instance per task code" by
joining on
`instance.end_time = t_max.max_end_time`. `end_time` is not unique, so when
two attempts
of the same task share an `end_time` the query returns **two rows for one
task code**.
On MySQL `t_ds_task_instance.end_time` is a `datetime` without fractional
seconds, so
every value is truncated to a whole second and a task which fails and is
retried quickly
ends up with both attempts carrying an identical `end_time`. The query
filters on
`state != 8` only, not on `flag`, so the invalidated previous attempt is
included too.
The only caller, `DependentExecute#dependResultByAllTaskOfWorkflowInstance`,
keys the
result by task code with `Collectors.toMap` and no merge function, so the
duplicate throws
`IllegalStateException: Duplicate key` and the dependency evaluation of a
DEPENDENT task
configured with "ALL tasks" fails.
## Brief change log
- `TaskInstanceMapper.xml#findLastTaskInstances`: resolve the last attempt
per task code
with `max(id)` and join on the primary key, which yields exactly one row
per task code.
- Kept the previous behaviour of ignoring attempts which have not finished,
by filtering
`end_time is not null` in the sub-query. Previously this was implicit —
`max(end_time)`
ignores NULLs and `instance.end_time = t_max.max_end_time` never matches a
NULL.
- Added a `TaskInstanceDaoImplTest` case covering two attempts sharing an
`end_time`.
### A note on `max(id)` vs `max(end_time)`
`t_ds_task_instance.id` is an auto-increment primary key, so the highest id
for a
(`workflow_instance_id`, `task_code`) pair is the most recently created
attempt. A retry
row is only inserted after the previous attempt has ended, so for retries
the latest id
and the latest `end_time` identify the same row — but only `id` is unique,
so only `id`
can guarantee a single row. This also removes the non-aggregated
`workflow_instance_id`
from the `group by task_code` sub-query, which is friendlier to
`ONLY_FULL_GROUP_BY` on
MySQL.
Happy to switch to a different tie-break if maintainers prefer to keep
`end_time` as the
ordering key.
## Verify this pull request
This change added tests and can be verified as follows:
- Added
`TaskInstanceDaoImplTest#queryLastTaskInstanceListIntervalInWorkflowInstanceWhenAttemptsShareTheSameEndTime`,
which inserts a `FAILURE` attempt and its `SUCCESS` retry with the same
`end_time` and
asserts a single row — the last attempt — is returned.
```bash
./mvnw -pl dolphinscheduler-dao -am clean test \
-Dtest=TaskInstanceDaoImplTest \
-Dsurefire.failIfNoSpecifiedTests=false
```
Verified locally:
- The new test fails on `dev` with `expected: <1> but was: <2>` and passes
with this
change.
- The two pre-existing `TaskInstanceDaoImplTest` cases still pass, so the
"last attempt"
semantics are unchanged for the normal, non-tied case.
- The full `dolphinscheduler-dao` suite passes: 284 tests, 0 failures, 0
errors.
- `./mvnw -pl dolphinscheduler-dao spotless:check` passes.
## Pull Request Notice
[Pull Request
Notice](https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)
If your pull request contains incompatible change, you should also add it to
`docs/docs/en/guide/upgrade/incompatible.md`
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
