[
https://issues.apache.org/jira/browse/FLINK-40281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40281:
-----------------------------------
Labels: pull-request-available (was: )
> Improve HTTP sink failure observability
> ---------------------------------------
>
> Key: FLINK-40281
> URL: https://issues.apache.org/jira/browse/FLINK-40281
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / HTTP
> Reporter: Bharath Reddy Gunapati
> Priority: Major
> Labels: pull-request-available
>
> The HTTP lookup source exposes useful operational signals for HTTP calls,
> including retry-related metrics and metadata-driven error visibility. The
> HTTP sink currently has only
> basic failure observability: failed records increment the generic sink
> send-error counter, logs contain limited failure context, and the
> post-request callback receives a null
> response for client-side exceptions without the original cause.
> As the sink gains configurable response classification and retry behavior,
> operators need clearer visibility into why writes fail or retry. This ticket
> improves HTTP sink observability for retryable responses, fatal responses,
> ignored responses, retry exhaustion, and client-side exceptions.
> Proposed Scope
> - Add HTTP sink metrics for retryable response failures, fatal response
> failures, ignored responses, retry exhaustion, and client-side exceptions.
> - Add retry-attempt visibility, for example a retry attempt counter.
> - Add status-code visibility, either per status code or grouped by status
> class.
> - Improve sink failure logs to include endpoint, HTTP method, status code
> where available, batch size/request entry count, and retry attempt.
> - Improve callback/failure reporting so client-side exceptions are
> observable instead of only appearing as a null response.
> - Document the new metrics and failure observability behavior.
> Acceptance Criteria
> - HTTP sink exposes metrics that distinguish retryable response failures,
> fatal response failures, ignored responses, retry exhaustion, and client-side
> exceptions.
> - Sink logs include enough context to diagnose failed writes without
> enabling full request/response body logging.
> - Batch mode logs/metrics identify the number of request entries affected
> by a single HTTP response.
> - Client-side exceptions are observable through logs and/or
> callback/failure reporting with the original cause preserved.
> - Existing sink behavior remains unchanged unless users opt into any new
> callback behavior.
> - Unit tests cover metrics/logging or callback behavior for retryable,
> fatal, ignored, exhausted, and exception paths.
> - Documentation describes the new observability signals.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
