Thundercloud12 commented on issue #9127:
URL: https://github.com/apache/devlake/issues/9127#issuecomment-5741669112
@klesh hey! so I’ve gone through the discussion thread as well as this issue.
The main problem seems to be this check in `cicd_run_collector.go`:
```go
if run.Status != "completed" {
continue
}
```
This drops `in_progress` runs, while `CollectorStateManager.Close()`
advances `LatestSuccessStart` to `until`, so those runs may never be revisited.
I see two possible approaches:
1. **Low-water-mark:** persist unfinished runs and, before each sync, query
the oldest unfinished run and move `windowStart` back to it. This keeps the
shared watermark untouched.
2. **Recheck by run ID:** since `GithubRun` is already keyed by
`ConnectionId + RepoId + ID` and GitHub supports fetching a specific workflow
run by `run_id`, we could keep the normal incremental window and only recheck
previously `in_progress` runs by ID before continuing with the new window.
The second approach seems worth considering since it avoids re-fetching the
entire historical window just to update a small number of unfinished runs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
