ShivamSharma43 opened a new issue, #9358:
URL: https://github.com/apache/devlake/issues/9358

   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/devlake/issues?q=is%3Aissue) and found no 
similar issues.
   
   ### What happened
   
   The GitHub `extractApiIssues` and GitHub GraphQL `extractIssues` subtasks 
report success without extracting anything when their setup fails.
   
   Both drop the error from `NewIssueRegexes`, which compiles the issue regexes 
from the scope config (`plugins/github/tasks/issue_extractor.go`, 
`plugins/github_graphql/tasks/issue_extractor.go`):
   
   ```go
   issueRegexes, err := NewIssueRegexes(config)
   if err != nil {
        return nil
   }
   ```
   
   The GraphQL extractor does the same with the error from `getMilestoneMap`, 
which reads `_tool_github_milestones`:
   
   ```go
   milestoneMap, err := getMilestoneMap(db, data.Options.GithubId, 
data.Options.ConnectionId)
   if err != nil {
        return nil
   }
   ```
   
   So an invalid regex in the scope config (for example `issueSeverity: 
"severity/(.*"`) or a failed milestone query makes the subtask return `nil`. 
The pipeline marks it successful, `_tool_github_issues` isn't updated, and the 
converters run on old data. No error is logged, so the user doesn't learn that 
the regex is wrong.
   
   The PR extractors of both plugins return the same kind of error (`regexp 
Compile prType failed`), and so does the GitLab issue extractor.
   
   ### What do you expect to happen
   
   The subtask fails with the setup error, e.g. `regexp Compile issueSeverity 
failed`, like the PR extractors.
   
   ### How to reproduce
   
   Set an invalid regex such as `severity/(.*` as `issueSeverity` in a GitHub 
scope config and collect the repo. `Extract Issues` succeeds and no issues are 
extracted. A unit test that calls `ExtractApiIssues` or `ExtractIssues` with 
that scope config gets `nil` back.
   
   ### Anything else
   
   Proposed fix (I have a PR ready): `return err` instead of `return nil` in 
the three places.
   
   ### Version
   
   main
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to