ShivamSharma43 opened a new pull request, #9341:
URL: https://github.com/apache/devlake/pull/9341
### Summary
The Taiga `collectEpics`, `collectIssues`, `collectTasks` and
`collectUserStories` subtasks only collected the first 30 items of a project.
Their `Query` sent only `project`. Taiga list endpoints are paginated
(`PAGINATE_BY: 30` in taiga-back) unless `x-disable-pagination` is sent, and
return page 1 when no `page` is given. Each response had 30 items, which is
less than `PageSize: 1000`, so the `ApiCollector` stopped after one page. The
collector fetched pages 1..10 in parallel and all ten requests were identical,
so the raw tables also held the same 30 rows ten times.
Changes:
- `tasks/shared.go` (new):
- `taigaProjectListQuery` sends `project`, `page` and `page_size`. The
page size is 1000, Taiga's `MAX_PAGINATE_BY`.
- `getNextTaigaPage` ends the collection when the response has no
`X-Pagination-Next` header. Taiga answers a page past the last one with 404, so
a full last page must not lead to another request. Setting
`GetNextPageCustomData` also makes the collector fetch pages one at a time
instead of guessing 10 pages in parallel.
- The four collectors use these two helpers. Request URLs, raw tables and
extractors are unchanged.
- `tasks/shared_test.go` (new): checks the query params, and checks that the
next-page function stops with and without the header.
### Does this close any open issues?
Closes #9340
### Screenshots
N/A
### Other Information
- `go test ./plugins/taiga/tasks/...`: pass. The query test fails on the old
query (`page` and `page_size` are empty).
- `go vet ./plugins/taiga/...` and `gofmt`: clean. The e2e tests need
`E2E_DB_URL` and don't cover the collectors.
- No open PR touches the taiga plugin.
- Suggested label: `pr-type/bug-fix`.
- AI assistance disclosure: this change was prepared with the help of Claude
Code (Claude Opus 5.5). I reviewed the diff and the taiga-back pagination code
it relies on (`taiga/base/api/pagination.py`). The commit carries a
`Generated-by:` trailer per the ASF generative tooling guidance.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
