caioq opened a new issue, #7851: URL: https://github.com/apache/incubator-devlake/issues/7851
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar issues. ### What happened I'm having some issues with getting Pull Requests using Github datasource. There are pull requests that aren't being updated completely. I noticed this when PRs that were already merged still didn't have the merged_date and closed_date fields filled in and the status was still OPEN. I already have tried using the blueprint in advanced and default mode. I'm using GraphQL Github api. ### What do you expect to happen All Pull Requests being successfully updated ### How to reproduce Create a blueprint on normal or advanced mode and check for next executions if all the pull requests are being updated. ### Anything else Part of the logs of the stage ``` time="2024-08-01 12:32:19" level=info msg=" [pipeline service] [pipeline #36] [task #3456] [api async client] creating scheduler for api \"https://api.github.com/\";, number of workers: 20, 14500 reqs / 1h0m0s (interval: 248.275862ms)" time="2024-08-01 12:32:19" level=info msg=" [pipeline service] [pipeline #36] [task #3456] github graphql init success with remaining 12500/12500 and will reset at 2024-08-01 13:32:19 +0000 UTC" time="2024-08-01 12:32:19" level=info msg=" [pipeline service] [pipeline #36] [task #3456] total step: 17" time="2024-08-01 12:32:19" level=info msg=" [pipeline service] [pipeline #36] [task #3456] executing subtask Collect Pull Requests" time="2024-08-01 12:32:19" level=info msg=" [pipeline service] [pipeline #36] [task #3456] [Collect Pull Requests] start graphql collection" time="2024-08-01 12:32:19" level=info msg=" [pipeline service] [pipeline #36] [task #3456] [Collect Pull Requests] get data from _raw_github_graphql_prs where params={\"ConnectionId\":1,\"Name\":\"XXX\"} and got 39" time="2024-08-01 12:32:19" level=info msg=" [pipeline service] [pipeline #36] [task #3456] [Collect Pull Requests] existing data parser return ErrFinishCollect, but skip. rawId: #1" time="2024-08-01 12:32:19" level=info msg=" [pipeline service] [pipeline #36] [task #3456] [Collect Pull Requests] finished records: 1" time="2024-08-01 12:32:19" level=info msg=" [pipeline service] [pipeline #36] [task #3456] [Collect Pull Requests] existing data parser return ErrFinishCollect, but skip. rawId: #2" time="2024-08-01 12:32:19" level=info msg=" [pipeline service] [pipeline #36] [task #3456] [Collect Pull Requests] existing data parser return ErrFinishCollect, but skip. rawId: #3" time="2024-08-01 12:32:20" level=info msg=" [pipeline service] [pipeline #36] [task #3456] [Collect Pull Requests] collector finish by parser, rawId: #25942" time="2024-08-01 12:32:20" level=info msg=" [pipeline service] [pipeline #36] [task #3456] [Collect Pull Requests] ended api collection without error" time="2024-08-01 12:32:20" level=info msg=" [pipeline service] [pipeline #36] [task #3456] finished step: 1 / 17" time="2024-08-01 12:32:20" level=info msg=" [pipeline service] [pipeline #36] [task #3456] executing subtask Extract Pull Requests" time="2024-08-01 12:32:20" level=info msg=" [pipeline service] [pipeline #36] [task #3456] [Extract Pull Requests] get data from _raw_github_graphql_prs where params={\"ConnectionId\":1,\"Name\":\"XXX\"} and got 40" time="2024-08-01 12:32:21" level=info msg=" [pipeline service] [pipeline #36] [task #3456] [Extract Pull Requests] finished records: 1" time="2024-08-01 12:32:21" level=info msg=" [pipeline service] [pipeline #36] [task #3456] finished step: 2 / 17" time="2024-08-01 12:32:21" level=info msg=" [pipeline service] [pipeline #36] [task #3456] executing subtask Collect Users" time="2024-08-01 12:32:21" level=info msg=" [pipeline service] [pipeline #36] [task #3456] [Collect Users] start graphql collection" ``` ### Version v1.0.1-beta5 ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@devlake.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
