GitHub user tomm1990 added a comment to the discussion: [GitHub] Workflow runs
with in_progress status are skipped and not collected after completion
In `backend/plugins/github/tasks/cicd_run_collector.go` (tag `v1.0.2`) Collect
Workflow Runs only keeps runs with `status == "completed"`. Anything else
(including `in_progress`) is logged as `Skipping run{id: ..., number: ...} with
status ...` and discarded. Nothing is written for those IDs, so a later sync
cannot “update the existing unfinished record.”
that skip is expected in v1.0.2, and there is no toggle to “keep in_progress
and update later.”
Worse for multi-hour Actions: that collector is finalizable on `created_at`.
Incremental sync advances `since` from collected completed runs. A long run
that was skipped while in progress still has the old `created_at`. Once `since`
moves past that timestamp, a later completed sync may never walk back far
enough to see it again. That matches what you are seeing.
Current `main` still filters to completed only (it got quieter about the skip
log and uses `created=from..to` windows). So upgrading the official image alone
does not give you “store in_progress then patch on completion.”
Practical options without a custom image:
1. After long workflows finish (or on a schedule), run a **full sync** (or a
collect with `timeAfter` / time range that covers those runs’ `created_at`), so
the completed rows are listed again.
2. Keep incremental for day-to-day, and treat full/backfill sync as the safety
net for hour-scale jobs.
3. If you need true in-progress tracking, that is a product change (persist
non-completed + refresh by id/`updated_at`). Worth an issue against
`apache/devlake` with your ECS image tag and the skip log lines.
If you can paste one skipped run id and whether a full sync afterward creates
the `_tool_github_runs` row, that confirms the created_at window miss vs a
different filter problem.
GitHub link:
https://github.com/apache/devlake/discussions/9125#discussioncomment-18351581
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
