MartijnVisser commented on code in PR #57: URL: https://github.com/apache/flink-connector-http/pull/57#discussion_r4371210146
########## docs/content.zh/docs/connectors/datastream/http.md: ########## @@ -82,7 +83,21 @@ These options are specified on the builder using the setProperty method. | http.sink.writer.request.mode | optional | Sets the Http Sink request submission mode. Two modes are available: `single` and `batch`. Defaults to `batch` if not specified. | | http.sink.request.batch.size | optional | Applicable only for `http.sink.writer.request.mode = batch`. Sets number of individual events/requests that will be submitted as one HTTP request by HTTP sink. The default value is 500 which is same as HTTP Sink `maxBatchSize` | - +### Retries and handling errors (Sink) +Retries are disabled by default: `http.sink.max-retries` defaults to `0`, so the first failed request fails the job. + +When `http.sink.max-retries` is greater than `0`, the sink retries requests that fail before a response is received (for +example, connection errors) and responses whose status code is listed in `http.sink.retry-codes`. The delay between +retries is set by `http.sink.retry-strategy.type` and the matching fixed-delay or exponential-delay options. Any other +response that is not successful or ignored is not retried and fails the job. A request that still fails with a retryable +error after the last retry also fails the job. + +In batch submission mode, each HTTP request carries several records and gets one response status, which applies to every Review Comment: Not today: the sink only reads the status code. Per-record results need a parser for the endpoint's response body, out of scope for this ticket. A partly applied batch is resent in full. ########## docs/content/docs/connectors/table/http.md: ########## @@ -633,6 +634,22 @@ If either legacy property is set together with `http.sink.success-codes` or `htt - `http.sink.retry-codes` (default `500,503,504`) for retryable responses - `http.sink.ignored-response-codes` for responses that should be treated as successful without retrying +### Retries and handling errors (Sink) +Retries are disabled by default: `http.sink.max-retries` defaults to `0`, so the first failed request fails the job. + +When `http.sink.max-retries` is greater than `0`, the sink retries requests that fail before a response is received (for +example, connection errors) and responses whose status code is listed in `http.sink.retry-codes`. The delay between Review Comment: With the legacy `http.sink.error.code` properties, no status code is retried, only failures without a response. I think that needs a sentence here, on all four pages. ########## docs/content.zh/docs/connectors/table/http.md: ########## @@ -632,6 +633,22 @@ If either legacy property is set together with `http.sink.success-codes` or `htt - `http.sink.retry-codes` (default `500,503,504`) for retryable responses - `http.sink.ignored-response-codes` for responses that should be treated as successful without retrying +### Retries and handling errors (Sink) +Retries are disabled by default: `http.sink.max-retries` defaults to `0`, so the first failed request fails the job. + +When `http.sink.max-retries` is greater than `0`, the sink retries requests that fail before a response is received (for +example, connection errors) and responses whose status code is listed in `http.sink.retry-codes`. The delay between +retries is set by `http.sink.retry-strategy.type` and the matching fixed-delay or exponential-delay options. Any other +response that is not successful or ignored is not retried and fails the job. A request that still fails with a retryable +error after the last retry also fails the job. + +In batch submission mode, each HTTP request carries several records and gets one response status, which applies to every +record in that request. A retryable response retries all of those records, and a fatal response or an exhausted retry +fails all of them. The sink does not retry individual records inside a batch. Use an idempotent endpoint if retried +records must not be applied twice. When one flush is split into several HTTP requests (because of +`http.sink.request.batch.size` or different HTTP methods), each response applies only to the records in its own request. +Retried records are batched again, so they can be grouped into different HTTP requests than in the first attempt. Review Comment: Checked locally with `http.sink.request.batch.size` 2 and three records: only the failed request's records are resent. Please add a test, nothing in the PR covers this yet. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
