This is an automated email from the ASF dual-hosted git repository.
klesh pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/devlake.git
The following commit(s) were added to refs/heads/main by this push:
new 739771bd2 feat(github): collect GitHub issue fields and map them onto
domain issues (#9038)
739771bd2 is described below
commit 739771bd2f0c9cca5c9cad6801c96ba6c3c86975
Author: vbhanuchander-lang <[email protected]>
AuthorDate: Wed Aug 12 09:53:01 2026 -0400
feat(github): collect GitHub issue fields and map them onto domain issues
(#9038)
* feat(github): collect GitHub issue fields and map them onto domain issues
GitHub issue fields are organization-level structured issue metadata that
went generally available on 2026-07-02. They are typed, mutually exclusive
within a field, and shared across every repository in the organization —
which is what teams currently approximate with `type:`-style labels.
Collects issue field values and lets a scope config map a field onto an
issue column, where it takes precedence over the existing label regexes.
- New table `_tool_github_issue_field_values`, one row per issue per field,
carrying a queryable text form of the value alongside the original JSON.
- New subtasks Collect/Extract Issue Field Values, both disabled by default
so existing pipelines are unaffected until a mapping is configured.
- New scope config keys issueFieldPriority, issueFieldSeverity,
issueFieldComponent, issueFieldStoryPoint and issueFieldDueDate, each
holding a field *name*.
The mapping is applied in the issue convertor rather than written back into
`_tool_github_issues`: the collector iterates that table to build its
request
URLs, so a subtask that both read and wrote it was a cycle in the subtask
graph. Converting instead also keeps the tool layer as raw GitHub truth and
avoids adding columns there.
A 404 from the field-values endpoint is treated as "no field values" so an
organization that has never configured issue fields, or a token that cannot
see them, does not fail the whole task.
* test(github): add an e2e dataflow test for issue field values
Covers extraction and the scope config mapping end to end against a real
database, which the unit tests could not reach.
Extraction asserts the value normalisation per data type: a single_select
resolving to its option name and colour, an integral number rendering as
"5" rather than "5.0", a fractional number keeping its precision, a
multi_select joining option names while keeping the raw JSON array, and a
null value producing an empty value.
Conversion asserts the mapping reaches the domain issue -- priority,
component, story point and due date -- and, for the case that matters,
that an unparseable value is skipped with a warning rather than failing
the task or writing a wrong value: issue #7 carries "soon" in a date
field and a null priority, and comes out with neither set while the other
issues are untouched.
* fix(github): register the issue field values table in GetTablesInfo
Test_GetPluginTablesInfo compares the plugin's declared tables against the
ones its migrations create, and the new table was missing from the list:
table_info_test.go:121: The following tables are not returned by the
TablesInfo method
_tool_github_issue_field_values
Adds GithubIssueFieldValue to Github.GetTablesInfo(). Verified inside the
mericodev/lake-builder image the unit-test job uses, since the plugins
package needs libgit2 to build.
---
.../plugins/github/e2e/issue_field_value_test.go | 95 ++++++++++
.../_raw_github_api_issue_field_values.csv | 9 +
.../_tool_github_issue_field_values.csv | 9 +
.../snapshot_tables/issues_with_field_values.csv | 27 +++
backend/plugins/github/impl/impl.go | 1 +
backend/plugins/github/models/issue_field_value.go | 53 ++++++
.../20260809_add_github_issue_fields.go | 71 ++++++++
.../github/models/migrationscripts/register.go | 1 +
backend/plugins/github/models/scope_config.go | 29 ++-
backend/plugins/github/tasks/issue_convertor.go | 16 +-
.../plugins/github/tasks/issue_field_mapping.go | 202 +++++++++++++++++++++
.../github/tasks/issue_field_mapping_test.go | 91 ++++++++++
.../github/tasks/issue_field_value_collector.go | 124 +++++++++++++
.../github/tasks/issue_field_value_extractor.go | 143 +++++++++++++++
.../tasks/issue_field_value_extractor_test.go | 97 ++++++++++
15 files changed, 956 insertions(+), 12 deletions(-)
diff --git a/backend/plugins/github/e2e/issue_field_value_test.go
b/backend/plugins/github/e2e/issue_field_value_test.go
new file mode 100644
index 000000000..63b987255
--- /dev/null
+++ b/backend/plugins/github/e2e/issue_field_value_test.go
@@ -0,0 +1,95 @@
+/*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package e2e
+
+import (
+ "testing"
+
+ "github.com/apache/incubator-devlake/core/models/domainlayer/ticket"
+ "github.com/apache/incubator-devlake/helpers/e2ehelper"
+ "github.com/apache/incubator-devlake/plugins/github/impl"
+ "github.com/apache/incubator-devlake/plugins/github/models"
+ "github.com/apache/incubator-devlake/plugins/github/tasks"
+)
+
+// Extraction of issue field values, and the scope config mapping that carries
them into the
+// domain issue.
+func TestIssueFieldValueDataFlow(t *testing.T) {
+ var plugin impl.Github
+ dataflowTester := e2ehelper.NewDataFlowTester(t, "github", plugin)
+
+ taskData := &tasks.GithubTaskData{
+ Options: &tasks.GithubOptions{
+ ConnectionId: 1,
+ Name: "panjf2000/ants",
+ GithubId: 134018330,
+ ScopeConfig: &models.GithubScopeConfig{
+ // Mapped by field name. "Effort" and "Target
date" are two of the four fields
+ // GitHub preconfigures for every organization.
+ IssueFieldPriority: "Priority",
+ IssueFieldComponent: "Squad",
+ IssueFieldStoryPoint: "Effort",
+ IssueFieldDueDate: "Target date",
+ },
+ },
+ }
+
+ dataflowTester.ImportCsvIntoRawTable(
+ "./raw_tables/_raw_github_api_issue_field_values.csv",
+ "_raw_github_api_issue_field_values")
+
+ // verify extraction
+ dataflowTester.FlushTabler(&models.GithubIssueFieldValue{})
+ dataflowTester.Subtask(tasks.ExtractApiIssueFieldValuesMeta, taskData)
+ dataflowTester.VerifyTableWithOptions(
+ models.GithubIssueFieldValue{},
+ e2ehelper.TableOptions{
+ CSVRelPath:
"./snapshot_tables/_tool_github_issue_field_values.csv",
+ TargetFields: []string{
+ "connection_id",
+ "issue_id",
+ "field_id",
+ "field_name",
+ "data_type",
+ "value",
+ "raw_value",
+ "option_color",
+ }},
+ )
+
+ // verify the mapping onto the domain issue
+
dataflowTester.ImportCsvIntoTabler("./raw_tables/_tool_github_issues.csv",
&models.GithubIssue{})
+ dataflowTester.FlushTabler(&ticket.Issue{})
+ dataflowTester.FlushTabler(&ticket.BoardIssue{})
+ dataflowTester.Subtask(tasks.ConvertIssuesMeta, taskData)
+ dataflowTester.VerifyTableWithOptions(
+ &ticket.Issue{},
+ e2ehelper.TableOptions{
+ CSVRelPath:
"./snapshot_tables/issues_with_field_values.csv",
+ TargetFields: []string{
+ "id",
+ "issue_key",
+ "priority",
+ "component",
+ "story_point",
+ "due_date",
+ },
+ IgnoreTypes: []interface{}{},
+ },
+ )
+}
diff --git
a/backend/plugins/github/e2e/raw_tables/_raw_github_api_issue_field_values.csv
b/backend/plugins/github/e2e/raw_tables/_raw_github_api_issue_field_values.csv
new file mode 100644
index 000000000..76588f564
--- /dev/null
+++
b/backend/plugins/github/e2e/raw_tables/_raw_github_api_issue_field_values.csv
@@ -0,0 +1,9 @@
+id,params,data,url,input,created_at
+1,"{""ConnectionId"":1,""Name"":""panjf2000/ants""}","{""issue_field_id"":123,""issue_field_name"":""Priority"",""node_id"":""IFV_1"",""data_type"":""single_select"",""value"":""Critical"",""single_select_option"":{""id"":1,""name"":""Critical"",""color"":""ff0000""},""multi_select_options"":null}",https://api.github.com/repos/panjf2000/ants/issues/5/issue-field-values?page=1&per_page=100,"{""Number"":5,""GithubId"":346842831}",2026-08-09
12:00:00.000000+00:00
+2,"{""ConnectionId"":1,""Name"":""panjf2000/ants""}","{""issue_field_id"":456,""issue_field_name"":""Effort"",""node_id"":""IFV_2"",""data_type"":""number"",""value"":5,""single_select_option"":null,""multi_select_options"":null}",https://api.github.com/repos/panjf2000/ants/issues/5/issue-field-values?page=1&per_page=100,"{""Number"":5,""GithubId"":346842831}",2026-08-09
12:00:00.000000+00:00
+3,"{""ConnectionId"":1,""Name"":""panjf2000/ants""}","{""issue_field_id"":789,""issue_field_name"":""Target
date"",""node_id"":""IFV_3"",""data_type"":""date"",""value"":""2024-12-31"",""single_select_option"":null,""multi_select_options"":null}",https://api.github.com/repos/panjf2000/ants/issues/5/issue-field-values?page=1&per_page=100,"{""Number"":5,""GithubId"":346842831}",2026-08-09
12:00:00.000000+00:00
+4,"{""ConnectionId"":1,""Name"":""panjf2000/ants""}","{""issue_field_id"":123,""issue_field_name"":""Priority"",""node_id"":""IFV_4"",""data_type"":""single_select"",""value"":""Low"",""single_select_option"":{""id"":4,""name"":""Low"",""color"":""0052cc""},""multi_select_options"":null}",https://api.github.com/repos/panjf2000/ants/issues/6/issue-field-values?page=1&per_page=100,"{""Number"":6,""GithubId"":347255859}",2026-08-09
12:00:00.000000+00:00
+5,"{""ConnectionId"":1,""Name"":""panjf2000/ants""}","{""issue_field_id"":456,""issue_field_name"":""Effort"",""node_id"":""IFV_5"",""data_type"":""number"",""value"":2.5,""single_select_option"":null,""multi_select_options"":null}",https://api.github.com/repos/panjf2000/ants/issues/6/issue-field-values?page=1&per_page=100,"{""Number"":6,""GithubId"":347255859}",2026-08-09
12:00:00.000000+00:00
+6,"{""ConnectionId"":1,""Name"":""panjf2000/ants""}","{""issue_field_id"":901,""issue_field_name"":""Squad"",""node_id"":""IFV_6"",""data_type"":""multi_select"",""value"":[""backend"",""urgent""],""single_select_option"":null,""multi_select_options"":[{""id"":2,""name"":""backend"",""color"":""0052cc""},{""id"":3,""name"":""urgent"",""color"":""ff6600""}]}",https://api.github.com/repos/panjf2000/ants/issues/6/issue-field-values?page=1&per_page=100,"{""Number"":6,""GithubId"":347255859}";
[...]
+7,"{""ConnectionId"":1,""Name"":""panjf2000/ants""}","{""issue_field_id"":789,""issue_field_name"":""Target
date"",""node_id"":""IFV_7"",""data_type"":""date"",""value"":""soon"",""single_select_option"":null,""multi_select_options"":null}",https://api.github.com/repos/panjf2000/ants/issues/7/issue-field-values?page=1&per_page=100,"{""Number"":7,""GithubId"":348630179}",2026-08-09
12:00:00.000000+00:00
+8,"{""ConnectionId"":1,""Name"":""panjf2000/ants""}","{""issue_field_id"":123,""issue_field_name"":""Priority"",""node_id"":""IFV_8"",""data_type"":""single_select"",""value"":null,""single_select_option"":null,""multi_select_options"":null}",https://api.github.com/repos/panjf2000/ants/issues/7/issue-field-values?page=1&per_page=100,"{""Number"":7,""GithubId"":348630179}",2026-08-09
12:00:00.000000+00:00
diff --git
a/backend/plugins/github/e2e/snapshot_tables/_tool_github_issue_field_values.csv
b/backend/plugins/github/e2e/snapshot_tables/_tool_github_issue_field_values.csv
new file mode 100644
index 000000000..e4f3dd4ee
--- /dev/null
+++
b/backend/plugins/github/e2e/snapshot_tables/_tool_github_issue_field_values.csv
@@ -0,0 +1,9 @@
+connection_id,issue_id,field_id,field_name,data_type,value,raw_value,option_color
+1,346842831,123,Priority,single_select,Critical,"""Critical""",ff0000
+1,346842831,456,Effort,number,5,5,
+1,346842831,789,Target date,date,2024-12-31,"""2024-12-31""",
+1,347255859,123,Priority,single_select,Low,"""Low""",0052cc
+1,347255859,456,Effort,number,2.5,2.5,
+1,347255859,901,Squad,multi_select,"backend,urgent","[""backend"",""urgent""]",
+1,348630179,123,Priority,single_select,,null,
+1,348630179,789,Target date,date,soon,"""soon""",
diff --git
a/backend/plugins/github/e2e/snapshot_tables/issues_with_field_values.csv
b/backend/plugins/github/e2e/snapshot_tables/issues_with_field_values.csv
new file mode 100644
index 000000000..f06f110f8
--- /dev/null
+++ b/backend/plugins/github/e2e/snapshot_tables/issues_with_field_values.csv
@@ -0,0 +1,27 @@
+id,issue_key,priority,component,story_point,due_date
+github:GithubIssue:1:346842831,5,Critical,,5,2024-12-31T00:00:00.000+00:00
+github:GithubIssue:1:347255859,6,Low,"backend,urgent",2.5,
+github:GithubIssue:1:348630179,7,,,,
+github:GithubIssue:1:356703393,10,,,,
+github:GithubIssue:1:364361014,12,,,,
+github:GithubIssue:1:381941219,17,,,,
+github:GithubIssue:1:382039050,18,,,,
+github:GithubIssue:1:382574800,20,,,,
+github:GithubIssue:1:388907811,21,,,,
+github:GithubIssue:1:401277739,22,,,,
+github:GithubIssue:1:402513849,24,,,,
+github:GithubIssue:1:405951301,25,,,,
+github:GithubIssue:1:413968505,26,,,,
+github:GithubIssue:1:419183961,27,,,,
+github:GithubIssue:1:419268851,28,,,,
+github:GithubIssue:1:424634533,29,,,,
+github:GithubIssue:1:429972115,31,,,,
+github:GithubIssue:1:433564955,32,,,,
+github:GithubIssue:1:434069015,33,,,,
+github:GithubIssue:1:435486645,34,,,,
+github:GithubIssue:1:461280653,35,,,,
+github:GithubIssue:1:462631417,37,,,,
+github:GithubIssue:1:472125082,38,,,,
+github:GithubIssue:1:483164833,42,,,,
+github:GithubIssue:1:483736247,43,,,,
+github:GithubIssue:1:484311063,44,,,,
diff --git a/backend/plugins/github/impl/impl.go
b/backend/plugins/github/impl/impl.go
index ada24da0d..f012077d2 100644
--- a/backend/plugins/github/impl/impl.go
+++ b/backend/plugins/github/impl/impl.go
@@ -103,6 +103,7 @@ func (p Github) GetTablesInfo() []dal.Tabler {
&models.GithubReviewer{},
&models.GithubRun{},
&models.GithubIssueAssignee{},
+ &models.GithubIssueFieldValue{},
&models.GithubScopeConfig{},
&models.GithubDeployment{},
&models.GithubRelease{},
diff --git a/backend/plugins/github/models/issue_field_value.go
b/backend/plugins/github/models/issue_field_value.go
new file mode 100644
index 000000000..2679532f7
--- /dev/null
+++ b/backend/plugins/github/models/issue_field_value.go
@@ -0,0 +1,53 @@
+/*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package models
+
+import (
+ "github.com/apache/incubator-devlake/core/models/common"
+)
+
+// GithubIssueFieldValue holds one issue field value for one issue.
+//
+// Issue fields are organization-level structured metadata that GitHub made
generally
+// available on 2026-07-02. Unlike labels they are typed, mutually exclusive
within a
+// field, and shared across every repository in the organization.
+//
+// Value carries a queryable text form of the value regardless of DataType:
+// - single_select: the selected option name
+// - multi_select: the selected option names, comma separated, in API order
+// - text: the text
+// - number: the number, formatted without a trailing ".0" when
integral
+// - date: the date as returned by the API (YYYY-MM-DD)
+//
+// RawValue keeps the original JSON so a value that does not fit the text form
+// (a multi_select array, for instance) is still recoverable downstream.
+type GithubIssueFieldValue struct {
+ ConnectionId uint64 `gorm:"primaryKey"`
+ IssueId int `gorm:"primaryKey;comment:GitHub issue id, matches
_tool_github_issues.github_id"`
+ FieldId int `gorm:"primaryKey"`
+ FieldName string `gorm:"type:varchar(255);index"`
+ DataType string `gorm:"type:varchar(50)"`
+ Value string `gorm:"type:text"`
+ RawValue string `gorm:"type:text"`
+ OptionColor string `gorm:"type:varchar(50)"`
+ common.NoPKModel
+}
+
+func (GithubIssueFieldValue) TableName() string {
+ return "_tool_github_issue_field_values"
+}
diff --git
a/backend/plugins/github/models/migrationscripts/20260809_add_github_issue_fields.go
b/backend/plugins/github/models/migrationscripts/20260809_add_github_issue_fields.go
new file mode 100644
index 000000000..f6480900f
--- /dev/null
+++
b/backend/plugins/github/models/migrationscripts/20260809_add_github_issue_fields.go
@@ -0,0 +1,71 @@
+/*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package migrationscripts
+
+import (
+ "github.com/apache/incubator-devlake/core/context"
+ "github.com/apache/incubator-devlake/core/errors"
+
"github.com/apache/incubator-devlake/core/models/migrationscripts/archived"
+ "github.com/apache/incubator-devlake/helpers/migrationhelper"
+)
+
+type githubIssueFieldValue20260809 struct {
+ ConnectionId uint64 `gorm:"primaryKey"`
+ IssueId int `gorm:"primaryKey"`
+ FieldId int `gorm:"primaryKey"`
+ FieldName string `gorm:"type:varchar(255);index"`
+ DataType string `gorm:"type:varchar(50)"`
+ Value string `gorm:"type:text"`
+ RawValue string `gorm:"type:text"`
+ OptionColor string `gorm:"type:varchar(50)"`
+ archived.NoPKModel
+}
+
+func (githubIssueFieldValue20260809) TableName() string {
+ return "_tool_github_issue_field_values"
+}
+
+type githubScopeConfig20260809 struct {
+ IssueFieldPriority string `gorm:"type:varchar(255)"`
+ IssueFieldSeverity string `gorm:"type:varchar(255)"`
+ IssueFieldComponent string `gorm:"type:varchar(255)"`
+ IssueFieldStoryPoint string `gorm:"type:varchar(255)"`
+ IssueFieldDueDate string `gorm:"type:varchar(255)"`
+}
+
+func (githubScopeConfig20260809) TableName() string {
+ return "_tool_github_scope_configs"
+}
+
+type addGithubIssueFields struct{}
+
+func (*addGithubIssueFields) Up(basicRes context.BasicRes) errors.Error {
+ return migrationhelper.AutoMigrateTables(
+ basicRes,
+ &githubIssueFieldValue20260809{},
+ &githubScopeConfig20260809{},
+ )
+}
+
+func (*addGithubIssueFields) Version() uint64 {
+ return 20260809000001
+}
+
+func (*addGithubIssueFields) Name() string {
+ return "add github issue field values table and scope config issue
field mappings"
+}
diff --git a/backend/plugins/github/models/migrationscripts/register.go
b/backend/plugins/github/models/migrationscripts/register.go
index 65f538f31..0a35123ed 100644
--- a/backend/plugins/github/models/migrationscripts/register.go
+++ b/backend/plugins/github/models/migrationscripts/register.go
@@ -58,5 +58,6 @@ func All() []plugin.MigrationScript {
new(addRefreshTokenFields),
new(modifyTokenExpiresAtToNullable),
new(addPrSizeExcludedFileExtensions),
+ new(addGithubIssueFields),
}
}
diff --git a/backend/plugins/github/models/scope_config.go
b/backend/plugins/github/models/scope_config.go
index a19f33db6..ae3fdf75c 100644
--- a/backend/plugins/github/models/scope_config.go
+++ b/backend/plugins/github/models/scope_config.go
@@ -26,16 +26,25 @@ import (
var _ plugin.ToolLayerScopeConfig = (*GithubScopeConfig)(nil)
type GithubScopeConfig struct {
- common.ScopeConfig `mapstructure:",squash" json:",inline"
gorm:"embedded"`
- PrType string
`mapstructure:"prType,omitempty" json:"prType" gorm:"type:varchar(255)"`
- PrComponent string
`mapstructure:"prComponent,omitempty" json:"prComponent"
gorm:"type:varchar(255)"`
- PrBodyClosePattern string
`mapstructure:"prBodyClosePattern,omitempty" json:"prBodyClosePattern"
gorm:"type:varchar(255)"`
- IssueSeverity string
`mapstructure:"issueSeverity,omitempty" json:"issueSeverity"
gorm:"type:varchar(255)"`
- IssuePriority string
`mapstructure:"issuePriority,omitempty" json:"issuePriority"
gorm:"type:varchar(255)"`
- IssueComponent string
`mapstructure:"issueComponent,omitempty" json:"issueComponent"
gorm:"type:varchar(255)"`
- IssueTypeBug string
`mapstructure:"issueTypeBug,omitempty" json:"issueTypeBug"
gorm:"type:varchar(255)"`
- IssueTypeIncident string
`mapstructure:"issueTypeIncident,omitempty" json:"issueTypeIncident"
gorm:"type:varchar(255)"`
- IssueTypeRequirement string
`mapstructure:"issueTypeRequirement,omitempty" json:"issueTypeRequirement"
gorm:"type:varchar(255)"`
+ common.ScopeConfig `mapstructure:",squash" json:",inline"
gorm:"embedded"`
+ PrType string `mapstructure:"prType,omitempty"
json:"prType" gorm:"type:varchar(255)"`
+ PrComponent string `mapstructure:"prComponent,omitempty"
json:"prComponent" gorm:"type:varchar(255)"`
+ PrBodyClosePattern string
`mapstructure:"prBodyClosePattern,omitempty" json:"prBodyClosePattern"
gorm:"type:varchar(255)"`
+ IssueSeverity string `mapstructure:"issueSeverity,omitempty"
json:"issueSeverity" gorm:"type:varchar(255)"`
+ IssuePriority string `mapstructure:"issuePriority,omitempty"
json:"issuePriority" gorm:"type:varchar(255)"`
+ IssueComponent string `mapstructure:"issueComponent,omitempty"
json:"issueComponent" gorm:"type:varchar(255)"`
+ IssueTypeBug string `mapstructure:"issueTypeBug,omitempty"
json:"issueTypeBug" gorm:"type:varchar(255)"`
+ IssueTypeIncident string `mapstructure:"issueTypeIncident,omitempty"
json:"issueTypeIncident" gorm:"type:varchar(255)"`
+ IssueTypeRequirement string
`mapstructure:"issueTypeRequirement,omitempty" json:"issueTypeRequirement"
gorm:"type:varchar(255)"`
+ // Issue field mappings. Each holds the *name* of a GitHub issue field
(organization-level
+ // structured metadata) whose value should populate the matching issue
column. When a
+ // mapping is set and the issue carries a value for that field, it
takes precedence over
+ // the label regexes above, which cannot express mutual exclusion or
span repositories.
+ IssueFieldPriority string
`mapstructure:"issueFieldPriority,omitempty" json:"issueFieldPriority"
gorm:"type:varchar(255)"`
+ IssueFieldSeverity string
`mapstructure:"issueFieldSeverity,omitempty" json:"issueFieldSeverity"
gorm:"type:varchar(255)"`
+ IssueFieldComponent string
`mapstructure:"issueFieldComponent,omitempty" json:"issueFieldComponent"
gorm:"type:varchar(255)"`
+ IssueFieldStoryPoint string
`mapstructure:"issueFieldStoryPoint,omitempty" json:"issueFieldStoryPoint"
gorm:"type:varchar(255)"`
+ IssueFieldDueDate string
`mapstructure:"issueFieldDueDate,omitempty" json:"issueFieldDueDate"
gorm:"type:varchar(255)"`
DeploymentPattern string
`mapstructure:"deploymentPattern,omitempty" json:"deploymentPattern"
gorm:"type:varchar(255)"`
ProductionPattern string
`mapstructure:"productionPattern,omitempty" json:"productionPattern"
gorm:"type:varchar(255)"`
EnvNamePattern string
`mapstructure:"envNamePattern,omitempty" json:"envNamePattern"
gorm:"type:varchar(255)"`
diff --git a/backend/plugins/github/tasks/issue_convertor.go
b/backend/plugins/github/tasks/issue_convertor.go
index 13e79529a..2cb60d1ff 100644
--- a/backend/plugins/github/tasks/issue_convertor.go
+++ b/backend/plugins/github/tasks/issue_convertor.go
@@ -42,8 +42,9 @@ var ConvertIssuesMeta = plugin.SubTaskMeta{
Description: "Convert tool layer table github_issues into domain
layer table issues",
DomainTypes: []string{plugin.DOMAIN_TYPE_TICKET},
DependencyTables: []string{
- models.GithubIssue{}.TableName(), // cursor
- models.GithubAccount{}.TableName(), // id generator
+ models.GithubIssue{}.TableName(), // cursor
+ models.GithubAccount{}.TableName(), // id generator
+ models.GithubIssueFieldValue{}.TableName(), // issue field
mappings
//models.GithubRepo{}.TableName(), // id generator, but
config not regard as dependency
RAW_ISSUE_TABLE},
ProductTables: []string{
@@ -60,6 +61,15 @@ func ConvertIssues(taskCtx plugin.SubTaskContext)
errors.Error {
accountIdGen := didgen.NewDomainIdGenerator(&models.GithubAccount{})
boardIdGen := didgen.NewDomainIdGenerator(&models.GithubRepo{})
+ // GitHub issue fields are organization-level structured metadata. When
the scope config
+ // maps one onto an issue column it wins over the label regexes, which
cannot express
+ // mutual exclusion and do not span repositories.
+ fieldMapping := resolveIssueFieldMapping(data.Options.ScopeConfig)
+ fieldValues, err := loadIssueFieldValues(taskCtx,
data.Options.ConnectionId, fieldMapping)
+ if err != nil {
+ return err
+ }
+
converter, err :=
api.NewStatefulDataConverter(&api.StatefulDataConverterArgs[models.GithubIssue]{
SubtaskCommonArgs: &api.SubtaskCommonArgs{
SubTaskContext: taskCtx,
@@ -68,6 +78,7 @@ func ConvertIssues(taskCtx plugin.SubTaskContext)
errors.Error {
ConnectionId: data.Options.ConnectionId,
Name: data.Options.Name,
},
+ SubtaskConfig: fieldMapping.asSubtaskConfig(),
},
Input: func(stateManager *api.SubtaskStateManager) (dal.Rows,
errors.Error) {
clauses := []dal.Clause{
@@ -102,6 +113,7 @@ func ConvertIssues(taskCtx plugin.SubTaskContext)
errors.Error {
Severity: issue.Severity,
Component: issue.Component,
}
+ applyIssueFields(taskCtx, fieldMapping, fieldValues,
issue, domainIssue)
if issue.AssigneeId != 0 {
domainIssue.AssigneeId =
accountIdGen.Generate(data.Options.ConnectionId, issue.AssigneeId)
}
diff --git a/backend/plugins/github/tasks/issue_field_mapping.go
b/backend/plugins/github/tasks/issue_field_mapping.go
new file mode 100644
index 000000000..aee0901a2
--- /dev/null
+++ b/backend/plugins/github/tasks/issue_field_mapping.go
@@ -0,0 +1,202 @@
+/*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package tasks
+
+import (
+ "strconv"
+ "strings"
+ "time"
+
+ "github.com/apache/incubator-devlake/core/dal"
+ "github.com/apache/incubator-devlake/core/errors"
+ "github.com/apache/incubator-devlake/core/models/domainlayer/ticket"
+ "github.com/apache/incubator-devlake/core/plugin"
+ "github.com/apache/incubator-devlake/plugins/github/models"
+)
+
+// issueFieldMapping is the resolved set of GitHub issue field names that
should populate
+// domain issue columns, taken from the scope config.
+//
+// The mapping is applied when converting to the domain layer rather than
written back into
+// _tool_github_issues: the collector iterates that table to build its request
URLs, so a
+// subtask that both reads and writes it would be a cycle in the subtask graph.
+type issueFieldMapping struct {
+ priority string
+ severity string
+ component string
+ storyPoint string
+ dueDate string
+}
+
+func resolveIssueFieldMapping(scopeConfig *models.GithubScopeConfig)
issueFieldMapping {
+ if scopeConfig == nil {
+ return issueFieldMapping{}
+ }
+ return issueFieldMapping{
+ priority: strings.TrimSpace(scopeConfig.IssueFieldPriority),
+ severity: strings.TrimSpace(scopeConfig.IssueFieldSeverity),
+ component: strings.TrimSpace(scopeConfig.IssueFieldComponent),
+ storyPoint: strings.TrimSpace(scopeConfig.IssueFieldStoryPoint),
+ dueDate: strings.TrimSpace(scopeConfig.IssueFieldDueDate),
+ }
+}
+
+func (m issueFieldMapping) isEmpty() bool {
+ return m.priority == "" && m.severity == "" && m.component == "" &&
+ m.storyPoint == "" && m.dueDate == ""
+}
+
+// names returns the distinct, lower-cased field names the mapping refers to.
+func (m issueFieldMapping) names() []string {
+ seen := map[string]bool{}
+ var out []string
+ for _, name := range []string{m.priority, m.severity, m.component,
m.storyPoint, m.dueDate} {
+ if name == "" {
+ continue
+ }
+ lower := strings.ToLower(name)
+ if !seen[lower] {
+ seen[lower] = true
+ out = append(out, lower)
+ }
+ }
+ return out
+}
+
+// asSubtaskConfig lets the stateful converter re-run in full when a mapping
changes, so
+// clearing or repointing one is not silently skipped by an incremental run.
+func (m issueFieldMapping) asSubtaskConfig() map[string]string {
+ return map[string]string{
+ "issueFieldPriority": m.priority,
+ "issueFieldSeverity": m.severity,
+ "issueFieldComponent": m.component,
+ "issueFieldStoryPoint": m.storyPoint,
+ "issueFieldDueDate": m.dueDate,
+ }
+}
+
+// issueFieldValues maps a GitHub issue id to its mapped field values, keyed
by lower-cased
+// field name.
+type issueFieldValues map[int]map[string]string
+
+// loadIssueFieldValues fetches the mapped fields for a connection in one
query. The result is
+// bounded by (issues x mapped fields), so this stays a single round trip
rather than one per
+// issue. Returns nil when nothing is mapped.
+func loadIssueFieldValues(
+ taskCtx plugin.SubTaskContext,
+ connectionId uint64,
+ mapping issueFieldMapping,
+) (issueFieldValues, errors.Error) {
+ if mapping.isEmpty() {
+ return nil, nil
+ }
+ db := taskCtx.GetDal()
+ var values []models.GithubIssueFieldValue
+ err := db.All(&values,
+ dal.From(models.GithubIssueFieldValue{}.TableName()),
+ dal.Where("connection_id = ? and LOWER(field_name) in ?",
connectionId, mapping.names()),
+ )
+ if err != nil {
+ return nil, err
+ }
+ byIssue := make(issueFieldValues, len(values))
+ for _, value := range values {
+ if value.Value == "" {
+ continue
+ }
+ fields, ok := byIssue[value.IssueId]
+ if !ok {
+ fields = map[string]string{}
+ byIssue[value.IssueId] = fields
+ }
+ fields[strings.ToLower(value.FieldName)] = value.Value
+ }
+ return byIssue, nil
+}
+
+// lookup returns the value for a mapped field name on one issue, reporting
false when the
+// mapping is unset or the issue carries no value for it.
+func (v issueFieldValues) lookup(issueId int, fieldName string) (string, bool)
{
+ if v == nil || fieldName == "" {
+ return "", false
+ }
+ fields, ok := v[issueId]
+ if !ok {
+ return "", false
+ }
+ text, ok := fields[strings.ToLower(fieldName)]
+ return text, ok
+}
+
+// dateLayouts covers the date form the API documents plus the full timestamp
form, since a
+// date field can round-trip either way.
+var dateLayouts = []string{"2006-01-02", time.RFC3339, "2006-01-02T15:04:05Z"}
+
+func parseFieldDate(text string) (*time.Time, error) {
+ var lastErr error
+ for _, layout := range dateLayouts {
+ parsed, err := time.Parse(layout, text)
+ if err == nil {
+ return &parsed, nil
+ }
+ lastErr = err
+ }
+ return nil, lastErr
+}
+
+// applyIssueFields overlays the mapped GitHub issue field values onto a
domain issue. Values
+// that do not parse into their target type are logged and skipped, leaving
whatever the label
+// regexes produced rather than writing a wrong value.
+func applyIssueFields(
+ taskCtx plugin.SubTaskContext,
+ mapping issueFieldMapping,
+ values issueFieldValues,
+ issue *models.GithubIssue,
+ domainIssue *ticket.Issue,
+) {
+ if values == nil || mapping.isEmpty() {
+ return
+ }
+ logger := taskCtx.GetLogger()
+
+ if text, ok := values.lookup(issue.GithubId, mapping.priority); ok {
+ domainIssue.Priority = text
+ }
+ if text, ok := values.lookup(issue.GithubId, mapping.severity); ok {
+ domainIssue.Severity = text
+ }
+ if text, ok := values.lookup(issue.GithubId, mapping.component); ok {
+ domainIssue.Component = text
+ }
+ if text, ok := values.lookup(issue.GithubId, mapping.storyPoint); ok {
+ if number, err := strconv.ParseFloat(text, 64); err == nil {
+ domainIssue.StoryPoint = &number
+ } else {
+ logger.Warn(nil, "issue #%d: field %q value %q is not a
number, story point left unset",
+ issue.Number, mapping.storyPoint, text)
+ }
+ }
+ if text, ok := values.lookup(issue.GithubId, mapping.dueDate); ok {
+ if parsed, err := parseFieldDate(text); err == nil {
+ domainIssue.DueDate = parsed
+ } else {
+ logger.Warn(nil, "issue #%d: field %q value %q is not a
date, due date left unset",
+ issue.Number, mapping.dueDate, text)
+ }
+ }
+}
diff --git a/backend/plugins/github/tasks/issue_field_mapping_test.go
b/backend/plugins/github/tasks/issue_field_mapping_test.go
new file mode 100644
index 000000000..6c159bf24
--- /dev/null
+++ b/backend/plugins/github/tasks/issue_field_mapping_test.go
@@ -0,0 +1,91 @@
+/*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package tasks
+
+import (
+ "testing"
+ "time"
+
+ "github.com/apache/incubator-devlake/plugins/github/models"
+ "github.com/stretchr/testify/assert"
+)
+
+func TestResolveIssueFieldMappingTrimsAndTolueratesNilConfig(t *testing.T) {
+ assert.True(t, resolveIssueFieldMapping(nil).isEmpty())
+
+ mapping := resolveIssueFieldMapping(&models.GithubScopeConfig{
+ IssueFieldPriority: " Priority ",
+ IssueFieldStoryPoint: "Effort",
+ })
+ assert.Equal(t, "Priority", mapping.priority)
+ assert.Equal(t, "Effort", mapping.storyPoint)
+ assert.False(t, mapping.isEmpty())
+}
+
+func TestIssueFieldMappingIsEmpty(t *testing.T) {
+ assert.True(t, issueFieldMapping{}.isEmpty())
+ assert.False(t, issueFieldMapping{priority: "Priority"}.isEmpty())
+ assert.False(t, issueFieldMapping{dueDate: "Target date"}.isEmpty())
+}
+
+func TestIssueFieldMappingNamesAreLowercasedAndDeduplicated(t *testing.T) {
+ mapping := issueFieldMapping{
+ priority: "Priority",
+ severity: "priority", // one field driving two columns
+ component: "",
+ storyPoint: "Effort",
+ dueDate: "Target Date",
+ }
+ assert.Equal(t, []string{"priority", "effort", "target date"},
mapping.names())
+}
+
+func TestIssueFieldValuesLookupIsCaseInsensitiveAndNilSafe(t *testing.T) {
+ var missing issueFieldValues
+ _, ok := missing.lookup(1, "Priority")
+ assert.False(t, ok, "nil map must not panic")
+
+ values := issueFieldValues{7: {"priority": "Critical"}}
+
+ text, ok := values.lookup(7, "PRIORITY")
+ assert.True(t, ok)
+ assert.Equal(t, "Critical", text)
+
+ _, ok = values.lookup(7, "")
+ assert.False(t, ok, "an unset mapping must never match")
+
+ _, ok = values.lookup(8, "Priority")
+ assert.False(t, ok, "another issue's values must not leak")
+}
+
+func TestParseFieldDateAcceptsTheDocumentedAndTimestampForms(t *testing.T) {
+ expected := time.Date(2024, 12, 31, 0, 0, 0, 0, time.UTC)
+
+ for _, text := range []string{"2024-12-31", "2024-12-31T00:00:00Z",
"2024-12-31T00:00:00+00:00"} {
+ parsed, err := parseFieldDate(text)
+ assert.NoError(t, err, text)
+ assert.Equal(t, expected, parsed.UTC(), text)
+ }
+}
+
+func TestParseFieldDateRejectsNonDates(t *testing.T) {
+ _, err := parseFieldDate("soon")
+ assert.Error(t, err)
+
+ _, err = parseFieldDate("")
+ assert.Error(t, err)
+}
diff --git a/backend/plugins/github/tasks/issue_field_value_collector.go
b/backend/plugins/github/tasks/issue_field_value_collector.go
new file mode 100644
index 000000000..17211911a
--- /dev/null
+++ b/backend/plugins/github/tasks/issue_field_value_collector.go
@@ -0,0 +1,124 @@
+/*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package tasks
+
+import (
+ "encoding/json"
+ "fmt"
+ "net/http"
+ "net/url"
+ "reflect"
+
+ "github.com/apache/incubator-devlake/core/dal"
+ "github.com/apache/incubator-devlake/core/errors"
+ "github.com/apache/incubator-devlake/core/plugin"
+ helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
+ "github.com/apache/incubator-devlake/plugins/github/models"
+)
+
+func init() {
+ RegisterSubtaskMeta(&CollectApiIssueFieldValuesMeta)
+}
+
+const RAW_ISSUE_FIELD_VALUE_TABLE = "github_api_issue_field_values"
+
+// SimpleIssue carries the two identifiers the field-value endpoint needs: the
number to
+// build the URL, and the id to key the extracted rows on.
+type SimpleIssue struct {
+ Number int
+ GithubId int
+}
+
+var CollectApiIssueFieldValuesMeta = plugin.SubTaskMeta{
+ Name: "Collect Issue Field Values",
+ EntryPoint: CollectApiIssueFieldValues,
+ EnabledByDefault: false,
+ Description: "Collect issue field values from the GitHub API,
supports both timeFilter and diffSync.",
+ DomainTypes: []string{plugin.DOMAIN_TYPE_TICKET},
+ DependencyTables: []string{models.GithubIssue{}.TableName()},
+ ProductTables: []string{RAW_ISSUE_FIELD_VALUE_TABLE},
+}
+
+func CollectApiIssueFieldValues(taskCtx plugin.SubTaskContext) errors.Error {
+ db := taskCtx.GetDal()
+ data := taskCtx.GetData().(*GithubTaskData)
+
+ apiCollector, err :=
helper.NewStatefulApiCollector(helper.RawDataSubTaskArgs{
+ Ctx: taskCtx,
+ Params: GithubApiParams{
+ ConnectionId: data.Options.ConnectionId,
+ Name: data.Options.Name,
+ },
+ Table: RAW_ISSUE_FIELD_VALUE_TABLE,
+ })
+ if err != nil {
+ return err
+ }
+
+ clauses := []dal.Clause{
+ dal.Select("number, github_id"),
+ dal.From(models.GithubIssue{}.TableName()),
+ dal.Where("repo_id = ? and connection_id = ?",
data.Options.GithubId, data.Options.ConnectionId),
+ }
+ if apiCollector.IsIncremental() && apiCollector.GetSince() != nil {
+ clauses = append(clauses, dal.Where("github_updated_at > ?",
apiCollector.GetSince()))
+ }
+
+ cursor, err := db.Cursor(clauses...)
+ if err != nil {
+ return err
+ }
+
+ iterator, err := helper.NewDalCursorIterator(db, cursor,
reflect.TypeOf(SimpleIssue{}))
+ if err != nil {
+ return err
+ }
+
+ err = apiCollector.InitCollector(helper.ApiCollectorArgs{
+ ApiClient: data.ApiClient,
+ PageSize: 100,
+ Input: iterator,
+
+ UrlTemplate: "repos/{{ .Params.Name }}/issues/{{ .Input.Number
}}/issue-field-values",
+
+ Query: func(reqData *helper.RequestData) (url.Values,
errors.Error) {
+ query := url.Values{}
+ query.Set("page", fmt.Sprintf("%v", reqData.Pager.Page))
+ query.Set("per_page", fmt.Sprintf("%v",
reqData.Pager.Size))
+ return query, nil
+ },
+ ResponseParser: func(res *http.Response) ([]json.RawMessage,
errors.Error) {
+ // An organization that has never configured issue
fields, or a token without
+ // visibility of them, gets 404 rather than an empty
list. Treat that as "this
+ // issue has no field values" so one such repository
cannot fail the whole task.
+ if res.StatusCode == http.StatusNotFound {
+ return nil, nil
+ }
+ var items []json.RawMessage
+ err := helper.UnmarshalResponse(res, &items)
+ if err != nil {
+ return nil, err
+ }
+ return items, nil
+ },
+ })
+ if err != nil {
+ return err
+ }
+ return apiCollector.Execute()
+}
diff --git a/backend/plugins/github/tasks/issue_field_value_extractor.go
b/backend/plugins/github/tasks/issue_field_value_extractor.go
new file mode 100644
index 000000000..4ec955df8
--- /dev/null
+++ b/backend/plugins/github/tasks/issue_field_value_extractor.go
@@ -0,0 +1,143 @@
+/*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package tasks
+
+import (
+ "encoding/json"
+ "strconv"
+ "strings"
+
+ "github.com/apache/incubator-devlake/core/errors"
+ "github.com/apache/incubator-devlake/core/plugin"
+ helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
+ "github.com/apache/incubator-devlake/plugins/github/models"
+)
+
+func init() {
+ RegisterSubtaskMeta(&ExtractApiIssueFieldValuesMeta)
+}
+
+var ExtractApiIssueFieldValuesMeta = plugin.SubTaskMeta{
+ Name: "Extract Issue Field Values",
+ EntryPoint: ExtractApiIssueFieldValues,
+ EnabledByDefault: false,
+ Description: "Extract raw issue field value data into the tool
layer table _tool_github_issue_field_values",
+ DomainTypes: []string{plugin.DOMAIN_TYPE_TICKET},
+ DependencyTables: []string{RAW_ISSUE_FIELD_VALUE_TABLE},
+ ProductTables: []string{models.GithubIssueFieldValue{}.TableName()},
+}
+
+type IssueFieldValueSelectOption struct {
+ Id int `json:"id"`
+ Name string `json:"name"`
+ Color string `json:"color"`
+}
+
+type IssueFieldValueResponse struct {
+ IssueFieldId int
`json:"issue_field_id"`
+ IssueFieldName string
`json:"issue_field_name"`
+ DataType string `json:"data_type"`
+ Value json.RawMessage `json:"value"`
+ SingleSelectOption *IssueFieldValueSelectOption
`json:"single_select_option"`
+ MultiSelectOptions []*IssueFieldValueSelectOption
`json:"multi_select_options"`
+}
+
+func ExtractApiIssueFieldValues(taskCtx plugin.SubTaskContext) errors.Error {
+ data := taskCtx.GetData().(*GithubTaskData)
+ extractor, err :=
helper.NewStatefulApiExtractor(&helper.StatefulApiExtractorArgs[IssueFieldValueResponse]{
+ SubtaskCommonArgs: &helper.SubtaskCommonArgs{
+ SubTaskContext: taskCtx,
+ Params: GithubApiParams{
+ ConnectionId: data.Options.ConnectionId,
+ Name: data.Options.Name,
+ },
+ Table: RAW_ISSUE_FIELD_VALUE_TABLE,
+ },
+ Extract: func(body *IssueFieldValueResponse, row
*helper.RawData) ([]any, errors.Error) {
+ if body.IssueFieldId == 0 {
+ return nil, nil
+ }
+ issue := &SimpleIssue{}
+ if err := errors.Convert(json.Unmarshal(row.Input,
issue)); err != nil {
+ return nil, err
+ }
+
+ fieldValue := &models.GithubIssueFieldValue{
+ ConnectionId: data.Options.ConnectionId,
+ IssueId: issue.GithubId,
+ FieldId: body.IssueFieldId,
+ FieldName: body.IssueFieldName,
+ DataType: body.DataType,
+ Value: displayValue(body),
+ RawValue: string(body.Value),
+ }
+ if body.SingleSelectOption != nil {
+ fieldValue.OptionColor =
body.SingleSelectOption.Color
+ }
+ return []any{fieldValue}, nil
+ },
+ })
+ if err != nil {
+ return err
+ }
+ return extractor.Execute()
+}
+
+// displayValue renders the API value as queryable text. Select options are
preferred over
+// the raw value because the option objects carry the canonical names.
+func displayValue(body *IssueFieldValueResponse) string {
+ if body.SingleSelectOption != nil {
+ return body.SingleSelectOption.Name
+ }
+ if len(body.MultiSelectOptions) > 0 {
+ names := make([]string, 0, len(body.MultiSelectOptions))
+ for _, option := range body.MultiSelectOptions {
+ if option != nil {
+ names = append(names, option.Name)
+ }
+ }
+ return strings.Join(names, ",")
+ }
+ return scalarValue(body.Value)
+}
+
+// scalarValue unwraps a JSON scalar into text. Numbers are rendered without a
trailing
+// ".0" so an effort of 5 reads as "5" rather than "5.0" in a dashboard.
+func scalarValue(raw json.RawMessage) string {
+ if len(raw) == 0 || string(raw) == "null" {
+ return ""
+ }
+ var asString string
+ if err := json.Unmarshal(raw, &asString); err == nil {
+ return asString
+ }
+ var asNumber float64
+ if err := json.Unmarshal(raw, &asNumber); err == nil {
+ return strconv.FormatFloat(asNumber, 'f', -1, 64)
+ }
+ var asBool bool
+ if err := json.Unmarshal(raw, &asBool); err == nil {
+ return strconv.FormatBool(asBool)
+ }
+ // Arrays of plain strings can arrive without option objects on
multi_select.
+ var asStrings []string
+ if err := json.Unmarshal(raw, &asStrings); err == nil {
+ return strings.Join(asStrings, ",")
+ }
+ return string(raw)
+}
diff --git a/backend/plugins/github/tasks/issue_field_value_extractor_test.go
b/backend/plugins/github/tasks/issue_field_value_extractor_test.go
new file mode 100644
index 000000000..38efc3676
--- /dev/null
+++ b/backend/plugins/github/tasks/issue_field_value_extractor_test.go
@@ -0,0 +1,97 @@
+/*
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package tasks
+
+import (
+ "encoding/json"
+ "testing"
+
+ "github.com/stretchr/testify/assert"
+)
+
+// The bodies below are the shapes documented for
+// GET /repos/{owner}/{repo}/issues/{issue_number}/issue-field-values.
+func TestDisplayValueAcrossDataTypes(t *testing.T) {
+ tests := []struct {
+ name string
+ body string
+ expected string
+ }{
+ {
+ name: "single_select prefers the option name",
+ body:
`{"issue_field_id":123,"issue_field_name":"Priority","data_type":"single_select",
+
"value":"Critical","single_select_option":{"id":1,"name":"Critical","color":"ff0000"}}`,
+ expected: "Critical",
+ },
+ {
+ name: "multi_select joins option names in API order",
+ body:
`{"issue_field_id":101,"issue_field_name":"Labels","data_type":"multi_select",
+ "value":["backend","urgent"],
+
"multi_select_options":[{"id":2,"name":"backend"},{"id":3,"name":"urgent"}]}`,
+ expected: "backend,urgent",
+ },
+ {
+ name: "text passes through",
+ body:
`{"issue_field_id":202,"issue_field_name":"Description","data_type":"text","value":"Fix
auth flow"}`,
+ expected: "Fix auth flow",
+ },
+ {
+ name: "integral number has no trailing decimal",
+ body:
`{"issue_field_id":456,"issue_field_name":"Effort","data_type":"number","value":5}`,
+ expected: "5",
+ },
+ {
+ name: "fractional number keeps its precision",
+ body:
`{"issue_field_id":456,"issue_field_name":"Effort","data_type":"number","value":2.5}`,
+ expected: "2.5",
+ },
+ {
+ name: "date passes through",
+ body:
`{"issue_field_id":789,"issue_field_name":"Target
Date","data_type":"date","value":"2024-12-31"}`,
+ expected: "2024-12-31",
+ },
+ {
+ name: "null value yields empty text",
+ body:
`{"issue_field_id":789,"issue_field_name":"Target
Date","data_type":"date","value":null}`,
+ expected: "",
+ },
+ {
+ name: "multi_select without option objects falls back
to the raw array",
+ body:
`{"issue_field_id":101,"issue_field_name":"Labels","data_type":"multi_select",
+ "value":["backend","urgent"]}`,
+ expected: "backend,urgent",
+ },
+ }
+
+ for _, tt := range tests {
+ t.Run(tt.name, func(t *testing.T) {
+ body := &IssueFieldValueResponse{}
+ assert.NoError(t, json.Unmarshal([]byte(tt.body), body))
+ assert.Equal(t, tt.expected, displayValue(body))
+ })
+ }
+}
+
+func TestScalarValueHandlesMissingAndUnknownShapes(t *testing.T) {
+ assert.Equal(t, "", scalarValue(nil))
+ assert.Equal(t, "", scalarValue(json.RawMessage("")))
+ assert.Equal(t, "", scalarValue(json.RawMessage("null")))
+ assert.Equal(t, "true", scalarValue(json.RawMessage("true")))
+ // An object we do not model is preserved verbatim rather than dropped.
+ assert.Equal(t, `{"a":1}`, scalarValue(json.RawMessage(`{"a":1}`)))
+}
