kotwal-itpro opened a new pull request, #9048: URL: https://github.com/apache/devlake/pull/9048
Closes #8949. ## Problem On large Jira instances, `jira:collectIssues` occasionally receives an otherwise-successful (2xx) response whose body cannot be decoded as JSON — typically a truncated body from a dropped connection, or (in rare misconfigured-proxy cases) an HTML body served with a 200. The current `ResponseParser` returns an error, which aborts the entire subtask and kills the whole Jira collection even though thousands of other pages succeeded. This is distinct from HTTP error statuses (`502/503/504`), which are already handled by the collector's retry logic — the request is specifically about parse failures on successful responses. ## Fix Introduce a `JIRA_SKIP_UNPARSEABLE_ISSUES` env var: - **default** (unset or anything other than `"true"`): existing behaviour — surface the parse error and fail the subtask. - **`"true"`** (case-insensitive): log a warning and skip the unparseable page so the rest of the collection completes. The warning includes body length and a 256-byte body prefix so operators can still investigate without hiding the problem. The V2 (Server) and V3 (Cloud) collectors previously duplicated the response-parsing closure. Factored the shared logic into a `parseIssuesResponse(logger)` helper so the new behaviour applies to both. ## Test Adds `Test_parseIssuesResponse` in `issue_collector_test.go` covering: - Happy path returns the expected issues, no warning. - Default (env var unset) surfaces the parse error, no warning. - `JIRA_SKIP_UNPARSEABLE_ISSUES=true` returns an empty non-nil slice and logs exactly one warning. - Case-insensitivity (`TRUE`). - Explicit `"false"` behaves like default. Uses `t.Setenv` for isolation. Full `plugins/jira/tasks` suite still passes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
