ShivamSharma43 opened a new issue, #9332: URL: https://github.com/apache/devlake/issues/9332
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/devlake/issues?q=is%3Aissue) and found no similar issues. ### What happened The Testmo `collectRuns`, `collectMilestones` and `collectAutomationRuns` subtasks only collect the first page (100 items) of a project. They all use `GetTotalPagesFromResponse` in `backend/plugins/testmo/tasks/shared.go`, which reads the page count from `meta.total_pages`: ```go body := &models.TestmoResponse{} // data, links, meta, errors ... if meta, ok := body.Meta.(map[string]interface{}); ok { if totalPages, exists := meta["total_pages"]; exists { ... } } return 1, nil ``` Testmo list responses have no `meta` object. The pagination fields are at the top level of the body: the Testmo REST API spec (`RunPage`, `MilestonePage`, `AutomationRunPage`) lists `page`, `prev_page`, `next_page`, `last_page`, `per_page`, `total` and `result`. The plugin's own remote scope API already reads this format (`ProjectPage` in `backend/plugins/testmo/api/remote_api.go` follows `next_page`). So `meta` is always nil, the function always returns 1, and the `ApiCollector` never requests page 2. ### What do you expect to happen All runs, milestones and automation runs of the project are collected, i.e. the collectors fetch pages 2..`last_page`. ### How to reproduce 1. Add a Testmo project with more than 100 runs (or milestones / automation runs) as a data scope. 2. Run the blueprint. 3. `_raw_testmo_runs` (and `_tool_testmo_runs`) hold only 100 rows for the project, while the project's `run_count` is higher. A unit test that passes `{"page": 1, "next_page": 2, "last_page": 3, "per_page": 100, "total": 250, "result": [...]}` to `GetTotalPagesFromResponse` gets 1 instead of 3. ### Anything else Proposed fix (I have a PR ready): read the top-level `last_page` in `GetTotalPagesFromResponse` (1 when it is null), plus a unit test. ### Version main ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
