dosubot[bot] commented on issue #7508:
URL:
https://github.com/apache/incubator-devlake/issues/7508#issuecomment-2129437190
<!-- Answer -->
The missing `labels` in the `devlake._raw_github_api_issues` table could be
due to several factors:
1. **Regex Configuration**: The `IssueRegexes` might not be correctly
configured to match the labels in the issues. If the regex patterns for
severity, component, priority, or type do not match any labels, those labels
will not be processed.
2. **Label Extraction Logic**: The logic in the `convertGithubLabels`
function might not be correctly extracting and storing the labels. Ensure that
the labels are being appended to the `results` slice and that the
`joinedLabels` are being correctly joined and assigned to `githubIssue.Type`.
3. **Database Schema**: The schema for the `GithubIssueLabel` table might
not be correctly defined or might be missing. Ensure that the table
`_tool_github_issue_labels` exists and is correctly defined to store the labels.
To resolve this issue, you can:
1. **Verify Regex Patterns**: Ensure that the regex patterns in the
`GithubScopeConfig` are correctly defined to match the labels you expect.
2. **Debug Label Extraction**: Add logging or debugging statements in the
`convertGithubLabels` function to verify that labels are being correctly
processed and appended to the `results` slice.
3. **Check Database Schema**: Verify that the `GithubIssueLabel` table is
correctly defined and exists in the database. The table should have the
following schema:
```go
package archived
import (
"github.com/apache/incubator-devlake/core/models/migrationscripts/archived"
)
// Please note that Issue Labels can also apply to Pull Requests.
// Pull Requests are considered Issues in GitHub.
type GithubIssueLabel struct {
ConnectionId uint64 `gorm:"primaryKey"`
IssueId int `gorm:"primaryKey;autoIncrement:false"`
LabelName string `gorm:"primaryKey;type:varchar(255)"`
archived.NoPKModel
}
func (GithubIssueLabel) TableName() string {
return "_tool_github_issue_labels"
}
```
Here is the relevant code for reference:
```go
func convertGithubLabels(issueRegexes *IssueRegexes, issue *IssuesResponse,
githubIssue *models.GithubIssue) ([]interface{}, errors.Error) {
var results []interface{}
var joinedLabels []string
for _, label := range issue.Labels {
results = append(results, &models.GithubIssueLabel{
ConnectionId: githubIssue.ConnectionId,
IssueId: githubIssue.GithubId,
LabelName: label.Name,
})
if issueRegexes.SeverityRegex != nil &&
issueRegexes.SeverityRegex.MatchString(label.Name) {
githubIssue.Severity = label.Name
}
if issueRegexes.ComponentRegex != nil &&
issueRegexes.ComponentRegex.MatchString(label.Name) {
githubIssue.Component = label.Name
}
if issueRegexes.PriorityRegex != nil &&
issueRegexes.PriorityRegex.MatchString(label.Name) {
githubIssue.Priority = label.Name
}
if issueRegexes.TypeRequirementRegex != nil &&
issueRegexes.TypeRequirementRegex.MatchString(label.Name) {
githubIssue.StdType = ticket.REQUIREMENT
} else if issueRegexes.TypeBugRegex != nil &&
issueRegexes.TypeBugRegex.MatchString(label.Name) {
githubIssue.StdType = ticket.BUG
} else if issueRegexes.TypeIncidentRegex != nil &&
issueRegexes.TypeIncidentRegex.MatchString(label.Name) {
githubIssue.StdType = ticket.INCIDENT
}
joinedLabels = append(joinedLabels, label.Name)
}
if len(joinedLabels) > 0 {
githubIssue.Type = strings.Join(joinedLabels, ",")
}
return results, nil
}
```
Additionally, ensure that the `ConvertIssueLabels` subtask is correctly
converting the labels from the tool layer to the domain layer. Here is the
relevant code for the `ConvertIssueLabels` subtask:
```go
func ConvertIssueLabels(taskCtx plugin.SubTaskContext) errors.Error {
db := taskCtx.GetDal()
data := taskCtx.GetData().(*GithubTaskData)
repoId := data.Options.GithubId
cursor, err := db.Cursor(
dal.From(&models.GithubIssueLabel{}),
dal.Join(`left join _tool_github_issues on
_tool_github_issues.github_id = _tool_github_issue_labels.issue_id`),
dal.Where("_tool_github_issues.repo_id = ? and
_tool_github_issues.connection_id = ?", repoId, data.Options.ConnectionId),
dal.Orderby("issue_id ASC"),
)
if err != nil {
return err
}
defer cursor.Close()
issueIdGen := didgen.NewDomainIdGenerator(&models.GithubIssue{})
converter, err := api.NewDataConverter(api.DataConverterArgs{
RawDataSubTaskArgs: api.RawDataSubTaskArgs{
Ctx: taskCtx,
Params: GithubApiParams{
ConnectionId: data.Options.ConnectionId,
Name: data.Options.Name,
},
Table: RAW_ISSUE_TABLE,
},
InputRowType: reflect.TypeOf(models.GithubIssueLabel{}),
Input: cursor,
Convert: func(inputRow interface{}) ([]interface{},
errors.Error) {
issueLabel := inputRow.(*models.GithubIssueLabel)
domainIssueLabel := &ticket.IssueLabel{
IssueId:
issueIdGen.Generate(data.Options.ConnectionId, issueLabel.IssueId),
LabelName: issueLabel.LabelName,
}
return []interface{}{
domainIssueLabel,
}, nil
},
})
if err != nil {
return err
}
return converter.Execute()
}
```
Ensure that the `issue.Labels` are being correctly iterated over and that
each label is being processed and appended to the `results` slice.
Additionally, verify that the `joinedLabels` are correctly joined and assigned
to `githubIssue.Type`. Also, make sure that the `GithubIssueLabel` table is
correctly defined and exists in the database.
Finally, ensure that the `issue_labels` table in the domain layer is
correctly defined to store the labels. The schema should be as follows:
```go
package ticket
import (
"github.com/apache/incubator-devlake/core/models/common"
)
// Please note that Issue Labels can also apply to Pull Requests.
// Pull Requests are considered Issues in GitHub.
type IssueLabel struct {
IssueId string `json:"id"
gorm:"primaryKey;type:varchar(255);comment:This key is generated based on
details from the original plugin"` // format: <Plugin>:<Entity>:<PK0>:<PK1>
LabelName string `gorm:"primaryKey;type:varchar(255)"`
common.NoPKModel
}
func (IssueLabel) TableName() string {
return "issue_labels"
}
```
By verifying these configurations and ensuring the correct processing of
labels, you can address the issue of missing `labels` in the
`devlake._raw_github_api_issues` table
<sup>[[1]](https://github.com/apache/incubator-devlake/blob/main/backend/plugins/github/tasks/issue_extractor.go)</sup><sup>[[2]](https://github.com/apache/incubator-devlake/blob/main/backend/plugins/github/models/migrationscripts/archived/issue_label.go)</sup><sup>[[3]](https://github.com/apache/incubator-devlake/blob/main/backend/plugins/github/tasks/issue_label_convertor.go)</sup><sup>[[4]](https://github.com/apache/incubator-devlake/blob/main/backend/core/models/domainlayer/ticket/issue_label.go)</sup>.
<!-- Dosu Comment Footer -->
<details>
<summary>
#### Details
</summary>
**About Dosu**
This response is meant to be useful and save you time. It is **not meant to
be a precise solution**, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is
irrelevant.
</details>
*To continue the conversation, mention @dosu.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@devlake.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
