hezyin opened a new issue, #3845: URL: https://github.com/apache/incubator-devlake/issues/3845
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar issues. ### What happened The closest bug I've found is this: https://github.com/apache/incubator-devlake/issues/3800 When I was debugging for a user in Slack, I found the downloaded full log doesn't match the error message from Config-UI and is clearly lacking some information. This is the error message the user saw on Config-UI, it has to do with git extractor failing to clone repos from GitLab instance with self-signed certificates:  The user downloaded the full log of the third run and shared it with me:  The full log tar.gz consists of three files. Please see their content below: * pipeling.log ``` time="2022-11-30 19:19:35" level=error msg=" [pipeline service] [pipeline #3] run tasks failed caused by: Error running task 10." ``` * task-9-1-1-gitlab.log ``` time="2022-11-30 19:18:59" level=info msg=" [pipeline service] [pipeline #3] [task #9] start executing task: 9" time="2022-11-30 19:18:59" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] start plugin" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] scheduler for api http://gitlab.redecorp.br/api/v4/ worker: 25, request: 18000, duration: 1h0m0s" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] total step: 28" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask collectApiProject" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiProject] start api collection" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiProject] finished records: 1" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiProject] end api collection without error" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 1 / 28" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask extractApiProject" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractApiProject] get data from _raw_gitlab_api_project where params={"ConnectionId":1,"ProjectId":13836} and got 1" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractApiProject] finished records: 1" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 2 / 28" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask collectApiIssues" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiIssues] start api collection" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiIssues] finished records: 1" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiIssues] end api collection without error" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 3 / 28" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask extractApiIssues" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractApiIssues] get data from _raw_gitlab_api_issues where params={"ConnectionId":1,"ProjectId":13836} and got 0" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 4 / 28" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask collectApiMergeRequests" time="2022-11-30 19:19:00" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequests] start api collection" time="2022-11-30 19:19:01" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequests] finished records: 1" time="2022-11-30 19:19:01" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequests] end api collection without error" time="2022-11-30 19:19:01" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 5 / 28" time="2022-11-30 19:19:01" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask extractApiMergeRequests" time="2022-11-30 19:19:01" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractApiMergeRequests] get data from _raw_gitlab_api_merge_requests where params={"ConnectionId":1,"ProjectId":13836} and got 76" time="2022-11-30 19:19:01" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractApiMergeRequests] finished records: 1" time="2022-11-30 19:19:01" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 6 / 28" time="2022-11-30 19:19:01" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask collectApiMergeRequestsNotes" time="2022-11-30 19:19:01" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequestsNotes] start api collection" time="2022-11-30 19:19:01" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequestsNotes] finished records: 1" time="2022-11-30 19:19:04" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequestsNotes] finished records: 17" time="2022-11-30 19:19:07" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequestsNotes] finished records: 32" time="2022-11-30 19:19:10" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequestsNotes] finished records: 47" time="2022-11-30 19:19:14" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequestsNotes] finished records: 63" time="2022-11-30 19:19:16" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequestsNotes] end api collection without error" time="2022-11-30 19:19:16" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 7 / 28" time="2022-11-30 19:19:16" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask extractApiMergeRequestsNotes" time="2022-11-30 19:19:16" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractApiMergeRequestsNotes] get data from _raw_gitlab_api_merge_request_notes where params={"ConnectionId":1,"ProjectId":13836} and got 442" time="2022-11-30 19:19:16" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractApiMergeRequestsNotes] finished records: 1" time="2022-11-30 19:19:16" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractApiMergeRequestsNotes] finished records: 1000" time="2022-11-30 19:19:16" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 8 / 28" time="2022-11-30 19:19:16" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask collectApiMergeRequestsCommits" time="2022-11-30 19:19:16" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequestsCommits] start api collection" time="2022-11-30 19:19:17" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequestsCommits] finished records: 1" time="2022-11-30 19:19:20" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequestsCommits] finished records: 17" time="2022-11-30 19:19:23" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequestsCommits] finished records: 33" time="2022-11-30 19:19:26" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequestsCommits] finished records: 48" time="2022-11-30 19:19:29" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequestsCommits] finished records: 64" time="2022-11-30 19:19:31" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiMergeRequestsCommits] end api collection without error" time="2022-11-30 19:19:31" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 9 / 28" time="2022-11-30 19:19:31" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask extractApiMergeRequestsCommits" time="2022-11-30 19:19:31" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractApiMergeRequestsCommits] get data from _raw_gitlab_api_merge_request_commits where params={"ConnectionId":1,"ProjectId":13836} and got 387" time="2022-11-30 19:19:31" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractApiMergeRequestsCommits] finished records: 1" time="2022-11-30 19:19:32" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractApiMergeRequestsCommits] finished records: 1000" time="2022-11-30 19:19:32" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 10 / 28" time="2022-11-30 19:19:32" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask collectApiPipelines" time="2022-11-30 19:19:32" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiPipelines] start api collection" time="2022-11-30 19:19:32" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiPipelines] finished records: 1" time="2022-11-30 19:19:33" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiPipelines] end api collection without error" time="2022-11-30 19:19:33" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 11 / 28" time="2022-11-30 19:19:33" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask extractApiPipelines" time="2022-11-30 19:19:33" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractApiPipelines] get data from _raw_gitlab_api_pipeline where params={"ConnectionId":1,"ProjectId":13836} and got 108" time="2022-11-30 19:19:33" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractApiPipelines] finished records: 1" time="2022-11-30 19:19:33" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 12 / 28" time="2022-11-30 19:19:33" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask collectApiJobs" time="2022-11-30 19:19:33" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiJobs] start api collection" time="2022-11-30 19:19:33" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiJobs] finished records: 1" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectApiJobs] end api collection without error" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 13 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask extractApiJobs" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractApiJobs] get data from _raw_gitlab_api_job where params={"ConnectionId":1,"ProjectId":13836} and got 0" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 14 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask enrichMrs" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [enrichMrs] finished records: 1" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 15 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask collectAccounts" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectAccounts] collect gitlab users" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectAccounts] start api collection" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectAccounts] finished records: 1" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [collectAccounts] end api collection without error" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 16 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask extractAccounts" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractAccounts] get data from _raw_gitlab_api_users where params={"ConnectionId":1,"ProjectId":13836} and got 20" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [extractAccounts] finished records: 1" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 17 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask convertAccounts" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [convertAccounts] finished records: 1" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 18 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask convertApiProject" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [convertApiProject] finished records: 1" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 19 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask convertApiMergeRequests" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [convertApiMergeRequests] finished records: 1" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 20 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask convertMergeRequestComment" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [convertMergeRequestComment] finished records: 1" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 21 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask convertApiMergeRequestsCommits" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [convertApiMergeRequestsCommits] finished records: 1" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 22 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask convertIssues" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 23 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask convertIssueLabels" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 24 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask convertMrLabels" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 25 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask convertPipelines" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [convertPipelines] finished records: 1" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 26 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask convertPipelineCommits" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] [convertPipelineCommits] finished records: 1" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 27 / 28" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] executing subtask convertJobs" time="2022-11-30 19:19:35" level=info msg=" [pipeline service] [pipeline #3] [task #9] [gitlab] finished step: 28 / 28" ``` * task-10-1-2-gitextractor.log ``` time="2022-11-30 19:18:59" level=info msg=" [pipeline service] [pipeline #3] [task #10] start executing task: 10" time="2022-11-30 19:18:59" level=info msg=" [pipeline service] [pipeline #3] [task #10] [gitextractor] start plugin" ``` It seems task-10-1-2-gitextractor.log does not contain the complete information, for example, not including the user rejected certificate error message. ### What do you expect to happen I expect task-10-1-2-gitextractor.log to contain more info for debugging. ### How to reproduce Please see above. ### Anything else _No response_ ### Version v0.14.2 ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
