antarasevich opened a new pull request, #8861:
URL: https://github.com/apache/incubator-devlake/pull/8861

   ## Summary
   
   Fix duplicate `ArgocdSyncOperation` records caused by ArgoCD exposing the 
same completed sync in two places in its API response: `status.history[]` and 
`status.operationState`.
   
   ## Problem
   
   ArgoCD includes the most recent completed sync in both:
   - `status.history[]` — the authoritative log of all past syncs, each with 
`deployedAt`
   - `status.operationState` — a mirror of the last operation, with `startedAt`
   
   The DevLake collector feeds both into the raw table. The extractor then 
processes each independently, using different timestamp fields as 
`DeploymentId`:
   
   | Source | Field used | Example value | DeploymentId |
   |---|---|---|---|
   | `history[id=30]` | `deployedAt` | `2026-04-16T09:02:16Z` | `1776330136` |
   | `operationState` | `startedAt` | `2026-04-16T09:02:15Z` | `1776330135` |
   
   Because `startedAt` and `deployedAt` differ by one second (the sync 
duration), the two records get different primary keys and both are persisted — 
resulting in duplicate deployment entries in DORA metrics and deployment 
frequency charts.
   
   ## Fix
   
   Extract the skip logic into a dedicated `isRedundantOperationState(phase 
string) bool` function and call it from inside `Extract` before processing any 
`operationState` row.
   
   A `Succeeded` `operationState` is always already present in 
`status.history[]` so it is skipped. In-flight (`Running`, `Terminating`) and 
failure (`Failed`, `Error`) states are not yet — or may never be — written to 
history, so they continue to be processed normally.
   
   ```go
   // isRedundantOperationState reports whether an operationState entry should 
be
   // skipped because it duplicates the most recent history entry.
   func isRedundantOperationState(phase string) bool {
       return phase == "Succeeded"
   }
   ```
   
   ```go
   // inside Extract:
   isOperationState := apiOp.Phase != ""
   
   if isOperationState && isRedundantOperationState(apiOp.Phase) {
       return nil, nil
   }
   ```
   
   ## Behaviour by phase
   
   | `operationState` phase | Already in `history[]`? | Action |
   |---|---|---|
   | `Succeeded` | yes | skip — history entry is authoritative |
   | `Running` / `Terminating` | no | keep — only source for in-flight sync |
   | `Failed` / `Error` | no | keep — failure may never reach history |
   | ` ` (empty) | n/a — history entry, not operationState | keep — guard never 
fires |
   
   ## Testing
   
   Added unit tests for `isRedundantOperationState` covering all phase 
variants. Verified against production ArgoCD API responses for multiple 
applications where `operationState.phase = Succeeded` — duplicate 
`ArgocdSyncOperation` records are eliminated after the fix.
   
   ## Related
   
   - Affects: `backend/plugins/argocd/tasks/sync_operation_extractor.go`
   - Reproduces with: any ArgoCD application whose last sync completed 
successfully (i.e. the vast majority of applications in a healthy cluster)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to