pballester opened a new issue, #9163: URL: https://github.com/apache/devlake/issues/9163
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/devlake/issues?q=is%3Aissue) and found no similar issues. ### What happened On a Tempo connection with team scopes, `collect_worklogs` never collects more than **1000 worklogs per team**, and they are always the **oldest** ones of a rolling 90-day window — recent worklogs never arrive. Seen on v1.0.3-beta15; the collector is unchanged on `main`. Evidence from `_raw_tempo_api_worklogs` after ~50 daily runs: - every request URL has `offset=0`, e.g. `https://api.tempo.io/4/worklogs/team/4?from=2026-06-25&limit=1000&offset=0&to=2026-09-23`, and each returns exactly 1000 results, i.e. there were more pages that were never requested; - `from` is always *today − 90 days*, regardless of the blueprint's `timeAfter` (2026-01-28 here); - resulting `_tool_tempo_worklogs` by start month: May 5040, Jun 5109, Jul 2208, Aug 461, Sep 46 — nothing before May, and a steady decline towards today, while the same teams log a roughly constant volume. Root causes in `backend/plugins/tempo/tasks/worklog_collector.go`: 1. **Pagination** — `GetTotalPages` computes the page count from `metadata.total`, but the Tempo v4 API does not return a total: per the official OpenAPI spec (`https://apidocs.tempo.io/tempo-openapi.yaml`), `PageableWorklog.metadata` is `PageableMetadata` = `count, limit, next, offset, previous`. `total` unmarshals as 0 → 0 pages → only the first page is fetched. 2. **Time range** — for team scopes the query uses `from = now − 90d` / `to = now` unless `fromDate`/`toDate` task options are set (the blueprint never sets them), ignoring the sync policy (`timeAfter`, incremental state). 3. **Shared state across teams** — the collector's raw params only carry `ConnectionId` (`TeamId` is always 0, although `TempoTeam.GetParams()` declares `ConnectionId` + `TeamId`), so all team scopes of a connection share one raw-data/collector-state key. With (2) fixed, the second team collected in a pipeline would run incrementally from the first team's start time. Separately, `convert_worklogs` emits ids that never join the jira domain layer: 4. `issue_worklogs.issue_id` is `jira:JiraIssues:<conn>:<id>` (plural), while the jira plugin generates `jira:JiraIssue:<conn>:<id>`; 5. `issue_worklogs.author_id` is the bare Atlassian account id instead of `jira:JiraAccount:<conn>:<id>`, so worklogs don't join `accounts` / `user_accounts`. ### What do you expect to happen All worklogs from `timeAfter` onwards are collected (then incrementally via `updatedFrom`), and `issue_worklogs` rows join `issues` and `accounts` like the jira plugin's own worklogs do. ### How to reproduce 1. Create a Tempo connection and add a team scope whose members log more than 1000 worklogs in 90 days. 2. Run a blueprint with `timeAfter` older than 90 days. 3. `SELECT url, COUNT(*) FROM _raw_tempo_api_worklogs GROUP BY url;` → one URL per run, `offset=0`, 1000 rows each; `SELECT MIN(start_date), MAX(start_date) FROM _tool_tempo_worklogs;` → starts 90 days back, sparse towards today. 4. `SELECT COUNT(*) FROM issue_worklogs w JOIN issues i ON i.id = w.issue_id WHERE w.id LIKE 'tempo:%';` → 0. ### Anything else Happens on every run. Worklogs collected by the jira plugin for time logged through Tempo carry the "Timesheets by Tempo" app as author, so the tempo plugin is the only source of per-person time — which makes this quite visible for anyone building per-team/per-person dashboards. ### Version v1.0.3-beta15 (collector unchanged on `main`) ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
