DoDiODev commented on PR #9015:
URL: https://github.com/apache/devlake/pull/9015#issuecomment-5165952362
Implemented as announced, plus the CI question from @klesh is now answered —
both in one update.
## 1. Targeted upgrade-path guard on **populated** tables
New: `backend/plugins/schema_e2e/migration_upgrade_path_test.go` →
`TestMigrationUpgradePathOnPopulatedTables`.
For every repair migration in this PR it
1. recreates the table **exactly as the buggy migration left it**,
2. inserts rows,
3. runs **only that repair script's** `Up()` (looked up through the plugin's
own `MigrationScripts()` **by version**, so the test breaks if a script is
removed or renumbered),
4. asserts: the columns were added, the rows survived, the table has a
primary key, auto-increment ids were backfilled with distinct non-zero values,
and a subsequent `INSERT` still works (i.e. the sequence/counter is in sync).
Covered (7 cases): `_tool_jira_sprint_reports`, `_tool_taiga_scope_configs`,
`_tool_teambition_scope_configs`, `_tool_testmo_scope_configs` and the three
`_tool_copilot_{enterprise,org,user}_ai_credit_usage` tables.
### Negative test — it really does catch the regression
Replacing the explicit `AUTO_INCREMENT` DDL in the teambition script with a
plain `AutoMigrate` (exactly the "someone simplifies this later" scenario
@vbhanuchander-lang described):
| engine | result |
| --- | --- |
| MySQL 8.4.10 | **FAIL** — `migration "add missing id/created_at/updated_at
columns …" failed on a populated "_tool_teambition_scope_configs"` (Error 1075)
|
| PostgreSQL 17.2 | **FAIL** — `table "_tool_teambition_scope_configs" has
no primary key after …` |
The Postgres case is the one the column-presence guard cannot see:
`AutoMigrate` happily adds `bigserial` *without* a key and reports success.
That gap is now closed by the primary-key assertion.
## 2. Rebase + a fourth instance of the same bug class
Rebased onto current `main` (10 upstream commits). The guard immediately
failed for `gh-copilot`: the three tables added by #9019 are missing all seven
credit-breakdown columns.
| table | missing |
| --- | --- |
| `_tool_copilot_enterprise_ai_credit_usage` | `gross_quantity`,
`discount_quantity`, `net_quantity`, `price_per_unit`, `gross_amount`,
`discount_amount`, `net_amount` |
| `_tool_copilot_org_ai_credit_usage` | same 7 |
| `_tool_copilot_user_ai_credit_usage` | same 7 |
Root cause: `20260708_add_ai_credit_usage_metrics.go` declares them via an
anonymous embedded struct whose **type name is unexported**
```go
creditUsageBreakdown20260708 `gorm:"embedded"`
```
GORM's schema parser skips anonymous fields of unexported types, so
`AutoMigrate` never created the columns — while the runtime models declare them
inline. The extractor would fail with `Unknown column 'gross_quantity' in
'field list'`: the Jira Sprint Report failure mode, one plugin over.
Fixed the same way as the other three — a new, additive migration
(`20260731_fix_ai_credit_usage_breakdown_columns.go`); the original script is
untouched. The two new plugins (`clickup`, `incidentio`) were registered in
`allGoPlugins()`, as `TestAllGoPluginsListed` demanded — the self-completing
property working as intended.
## 3. CI
The workflows on this PR sit at `action_required` (fork PR awaiting
maintainer approval), so I mirrored the same checks in my fork on an identical
tree — **all green**:
- `lint (go)`, `migration-script-lint`, `unit-test`, `config-ui`, ASF header
check, Grafana dashboard check
- `e2e (mysql)` (`make e2e-test-go-plugins` + `make e2e-test`), including:
- `--- PASS: TestMigrationSchema` (Jira-specific guard)
- `--- PASS: TestAllGoPluginsListed`
- `--- PASS: TestMigrationSchemaMatchesModels` — **44/44 plugins**
- `--- PASS: TestMigrationUpgradePathOnPopulatedTables` — **7/7 cases**
Locally also verified against **MySQL 8.4.10** and **PostgreSQL 17.2**:
51/51 subtests pass on both.
@klesh this should address the failing checks — happy to have the workflows
approved on the PR itself to confirm on your infrastructure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
