antarasevich opened a new pull request, #8861:
URL: https://github.com/apache/incubator-devlake/pull/8861
## Summary
Fix duplicate `ArgocdSyncOperation` records caused by ArgoCD exposing the
same completed sync in two places in its API response: `status.history[]` and
`status.operationState`.
## Problem
ArgoCD includes the most recent completed sync in both:
- `status.history[]` — the authoritative log of all past syncs, each with
`deployedAt`
- `status.operationState` — a mirror of the last operation, with `startedAt`
The DevLake collector feeds both into the raw table. The extractor then
processes each independently, using different timestamp fields as
`DeploymentId`:
| Source | Field used | Example value | DeploymentId |
|---|---|---|---|
| `history[id=30]` | `deployedAt` | `2026-04-16T09:02:16Z` | `1776330136` |
| `operationState` | `startedAt` | `2026-04-16T09:02:15Z` | `1776330135` |
Because `startedAt` and `deployedAt` differ by one second (the sync
duration), the two records get different primary keys and both are persisted —
resulting in duplicate deployment entries in DORA metrics and deployment
frequency charts.
## Fix
Extract the skip logic into a dedicated `isRedundantOperationState(phase
string) bool` function and call it from inside `Extract` before processing any
`operationState` row.
A `Succeeded` `operationState` is always already present in
`status.history[]` so it is skipped. In-flight (`Running`, `Terminating`) and
failure (`Failed`, `Error`) states are not yet — or may never be — written to
history, so they continue to be processed normally.
```go
// isRedundantOperationState reports whether an operationState entry should
be
// skipped because it duplicates the most recent history entry.
func isRedundantOperationState(phase string) bool {
return phase == "Succeeded"
}
```
```go
// inside Extract:
isOperationState := apiOp.Phase != ""
if isOperationState && isRedundantOperationState(apiOp.Phase) {
return nil, nil
}
```
## Behaviour by phase
| `operationState` phase | Already in `history[]`? | Action |
|---|---|---|
| `Succeeded` | yes | skip — history entry is authoritative |
| `Running` / `Terminating` | no | keep — only source for in-flight sync |
| `Failed` / `Error` | no | keep — failure may never reach history |
| ` ` (empty) | n/a — history entry, not operationState | keep — guard never
fires |
## Testing
Added unit tests for `isRedundantOperationState` covering all phase
variants. Verified against production ArgoCD API responses for multiple
applications where `operationState.phase = Succeeded` — duplicate
`ArgocdSyncOperation` records are eliminated after the fix.
## Related
- Affects: `backend/plugins/argocd/tasks/sync_operation_extractor.go`
- Reproduces with: any ArgoCD application whose last sync completed
successfully (i.e. the vast majority of applications in a healthy cluster)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]