dosubot[bot] commented on issue #8615: URL: https://github.com/apache/incubator-devlake/issues/8615#issuecomment-3415685323
<!-- Greeting --> Hi @FlomoN! I'm [Dosu](https://go.dosu.dev/dosubot) and I’m helping the incubator-devlake team. <!-- Answer --> You're correct—there's a pagination issue in the GitHub GraphQL job_collector. The code sets a page size of 20 when fetching jobs, and while it is designed to iterate over pages using the `hasNextPage` and `endCursor` from the GraphQL response, it appears that only the first page is being processed, which matches your observation of only 20 jobs per run being stored in the database. This results in missing jobs for workflow runs with more than 20 jobs, impacting DORA metrics and deployment detection. You can review and update the pagination logic in [`job_collector.go`](https://github.com/apache/incubator-devlake/blob/main/backend/plugins/github_graphql/tasks/job_collector.go) to ensure it continues fetching jobs until `hasNextPage` is false and all jobs are collected. If you're planning a PR, focus on verifying that the collector loops through all pages for each workflow run, using the `endCursor` to fetch subsequent pages. The relevant pagination handling is in the `getPageInfo` and `CollectJobs` functions. There are no configuration options to change the page size or pagination behavior in the current release, so a code fix is needed. Recent PRs have improved data correctness but haven't addressed this specific bug yet. Let me know if you need pointers to specific code sections or want to discuss implementation details! <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/0c116472-c6b2-40dc-ab33-0cb4275e861c?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/0c116472-c6b2-40dc-ab33-0cb4275e861c?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/0c116472-c6b2-40dc-ab33-0cb4275e861c?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/0c116472-c6b2-40dc-ab33-0cb4275e861c?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/0c116472-c6b2-40dc-ab33-0cb4275e861c?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/0c116472-c6b2-40dc-ab33-0cb4275e861c?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/0c116472-c6b2-40dc-ab33-0cb4275e861c?feedback_type=other)</sup> [](https://app.dosu.dev/b4e8e847-d479-4541-83a8-d88d83fea5c9/ask?utm_source=githu b) [](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/incubator-devlake/issues/8615) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
