breymander opened a new issue, #8789: URL: https://github.com/apache/incubator-devlake/issues/8789
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/incubator-devlake/issues?q=is%3Aissue) and found no similar issues. ### What happened The `collectOrgMetrics` subtask in the `gh-copilot` plugin fails immediately with `error parsing response` when collecting organization-level Copilot usage metrics via the `organization-1-day` report endpoint. The error occurs for every configured organization, blocks the entire pipeline, and prevents other tasks (Jenkins, Jira, etc.) from completing. **Error from logs:** ``` level=error msg=" [pipeline service] [pipeline #24] [task #5234] subtask collectOrgMetrics ended unexpectedly Wraps: (2) error parsing response from orgs/REDACTED_ORG/copilot/metrics/reports/organization-1-day Error types: (1) *hintdetail.withDetail (2) *errors.errorString" ``` The subtask completes in ~1 second (not a timeout), suggesting the GitHub API responds successfully but the response body cannot be unmarshalled into the expected `reportMetadataResponse` struct. **Environment:** - DevLake version: v1.0.3-beta10 - Database: MySQL (CloudSQL) - Authentication: Classic PAT with scopes `manage_billing:copilot`, `read:enterprise`, `read:org` - Copilot Metrics API policy: enabled - PAT owner: organization owner - Deployment: Helm on GKE ### What do you expect to happen The `collectOrgMetrics` subtask should either: 1. Successfully parse the response from `GET /orgs/{org}/copilot/metrics/reports/organization-1-day`, or 2. Provide a clear error message including the HTTP status code and response body when parsing fails (e.g., "unexpected response format, expected JSON object with download_links") ### How to reproduce 1. Deploy DevLake v1.0.3-beta10 2. Create a `gh-copilot` connection with a classic PAT and an organization (no enterprise) 3. Add the connection to a blueprint and trigger a pipeline 4. Observe `collectOrgMetrics` fails with "error parsing response" ### Analysis The `ResponseParser` in [`org_metrics_collector.go`](https://github.com/apache/incubator-devlake/blob/v1.0.3-beta10/backend/plugins/gh-copilot/tasks/org_metrics_collector.go) expects the response to unmarshal into a `reportMetadataResponse` struct (containing `DownloadLinks`). The GitHub API at `GET /orgs/{org}/copilot/metrics/reports/organization-1-day?day=YYYY-MM-DD` may be returning a response in a format that doesn't match this struct. Additionally, this error is fatal — it causes the entire pipeline to fail, blocking unrelated tasks like Jenkins and Jira collection that run in the same pipeline stage. ### Suggested improvements 1. Add verbose logging of the raw response body (or at least the HTTP status code and first N bytes) when parsing fails, to make debugging easier 2. Validate the response format and provide a clear error message (e.g., "unexpected response format, expected JSON object with download_links") ### Version v1.0.3-beta10 ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
