bujjibabukatta opened a new pull request, #9002:
URL: https://github.com/apache/devlake/pull/9002
Closes #8964
## Summary
The `gh-copilot` plugin pins the correct Copilot metrics API version
(`2026-03-10`) but is only partially implemented against it: several
response fields are dropped on extract, org-level rows lose their
identity, three endpoints aren't called at all, and a response-body
bug silently zeroes out both the enterprise and user report collectors.
## Bugfixes
**Double-read bug (2 occurrences).** `parseRawReportResponse` and
`parseUserMetricsReportResponse` each read `res.Body` once directly,
then again inside `parseReportMetadataResponse`. The second read
always came back empty, so `meta` was always `nil` and every
enterprise/user report was silently discarded — no error, no rows.
Fixed by reading once in both places.
**Org identity.** `ExtractOrgMetrics` hardcoded `EnterpriseId: ""` and
never set `OrganizationId`, so org-level rows in the shared daily
metrics table had no identity of their own. Now stamps
`OrganizationId` from the report (falling back to the connection's
configured org), and passes `EnterpriseId` through instead of forcing
it blank.
## Field gaps
| Field | Where |
|---|---|
| `daily/weekly/monthly_active_copilot_cloud_agent_users` | enterprise/org
daily metrics |
| `used_copilot_coding_agent`, `used_copilot_cloud_agent` | per-user daily
metrics |
| `ai_adoption_phase` / `totals_by_ai_adoption_phase` | new
`_tool_copilot_metrics_by_ai_adoption_phase` table |
| `pull_requests.copilot_suggestions_by_comment_type` | JSON blob column
(small cardinality, not worth a normalized table yet) |
| `totals_by_cli.last_known_cli_version`, `.avg_tokens_per_request` | shared
`CopilotCliMetrics` |
`ai_adoption_phase` field names (including the 2026-07-07 review-cycle
fields and 2026-06-26 `total_pull_requests_merged`) were checked
against GitHub's changelog posts.
## Missing endpoints
Implements `enterprise-28-day/latest`, `organization-28-day/latest`,
`users-28-day/latest`. These are single "latest" snapshot calls (no
day param, unlike the -1-day reports) whose payload wraps entries in
a `day_totals[]` array. Reuses the existing `enterpriseDayTotal`/
`userDailyReport` shapes for the entries; writes to **new, separate
tables** so this cadence can't silently overwrite -1-day rows for the
same `(connection, scope, day)`. Per-IDE/feature/language breakdowns
are out of scope for 28-day data in this pass — the -1-day breakdowns
already cover those dimensions.
> ⚠️ **Needs verification before merge:** the `day_totals[]` envelope
> shape is inferred from GitHub's endpoint naming/docs, not a captured
> sample response. There's a JSONL fallback for robustness, but please
> check against a real payload if you have API access.
## Migrations
- `20260720_add_copilot_metrics_gaps_v2.go` — new columns + AI
adoption phase table
- `20260720_add_copilot_28day_reports.go` — the two new 28-day tables
Migration struct field names were cross-checked against the real
model structs field-by-field to make sure `AutoMigrateTables` targets
the right columns.
## Testing
- [ ] `go build ./plugins/gh-copilot/...`
- [ ] `go vet ./plugins/gh-copilot/...`
- [ ] `go test ./plugins/gh-copilot/...`
- [ ] Migration dry run against a scratch DB
- [ ] Manual collection run against a real connection — confirm
non-empty daily metrics tables (regression test for the
double-read bug)
- [ ] Verify `day_totals[]` shape against a real 28-day report response
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
