GitHub user tomm1990 added a comment to the discussion: [GitHub] Workflow runs
with in_progress status are skipped and not collected after completion
@vasukiiyengar98 you are right that there is no Config UI toggle for a
*recurring* full/backfill sync. The cron you set (`0 */2 * * *`) always runs
**Collect Data (Default)** = incremental. that is why a 2h schedule alone
cannot recover a 3-4h run that was skipped while `in_progress`.
here is how to do the workaround on stock **v1.0.2** (Config UI + API):
### 1. Time Filter (`timeAfter`) - blueprint Sync Policy
In Config UI: open the **project / blueprint** → **Sync Policy** (or edit
policy).
- **Time Filter** is the blueprint `timeAfter` field (docs call it Time Filter).
- set it far enough back that a full refresh can still see those runs’
`created_at` (for 3-4h jobs, use at least several days, e.g. last 7 days / last
month - not “today only”).
that bound applies to both default collect and full refresh. it does **not** by
itself make incremental re-walk past the collector’s stored `since`.
### 2. One-shot full refresh from the UI
On the same blueprint detail page, use the collect control (not the cron):
- **Collect Data** = incremental (same as your schedule)
- **Collect Data in Full Refresh Mode** = delete/re-collect inside the Time
Filter window (`fullSync: true`)
run Full Refresh Mode after a long workflow finishes (or on a schedule you
own). that is the UI path that re-lists completed Actions runs whose
`created_at` would otherwise sit behind incremental `since`.
### 3. Recurring full refresh via API (recommended on ECS)
Config UI has no “full sync every N hours” cron. trigger it yourself against
the lake API:
```bash
curl -X POST "http://<devlake-api-host>/blueprints/<blueprintId>/trigger" \
-H "Content-Type: application/json" \
-d '{"skipCollectors": false, "fullSync": true}'
```
wire that to EventBridge / a small ECS scheduled task (e.g. every 6h or daily).
keep your existing 2h blueprint cron for normal incremental, and use this as
the safety net for hour-scale Actions.
`blueprintId` is in the Config UI URL / blueprint detail; API host is your
DevLake API service (same network as Config UI).
### 4. Future version / planned fix
there is still **no** public release or roadmap item I can point to that will
store `in_progress` and update on completion. I cannot honestly name a future
version for that. filing an issue on `apache/devlake` (GitHub plugin, image tag
`v1.0.2`, skip log lines, one run id missed by incremental but present after
full refresh) is the path to get it on the official backlog.
practical combo for your 3-4h prod workflows: keep `0 */2 * * *` + set Time
Filter ≥ several days + schedule `fullSync: true` triggers after those jobs (or
on a fixed cadence).
GitHub link:
https://github.com/apache/devlake/discussions/9125#discussioncomment-18366719
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
