bharathgunapati opened a new pull request, #54:
URL: https://github.com/apache/flink-connector-http/pull/54
## What is the purpose of the change
The HTTP sink currently treats non-success responses as generic request
failures.
This PR adds explicit sink response classification and retry handling so sink
behavior is closer to the lookup source status-code/retry model.
The sink can now classify HTTP responses as successful, ignored, retryable,
or
fatal, and retry retryable responses up to the configured retry limit.
## Brief change log
- Add configurable sink response code options for success, retry, and ignored
HTTP status codes.
- Add sink retry configuration and retry strategy options.
- Add response classification for success, ignored, retryable failure, and
fatal
failure.
- Retry retryable sink failures via `AsyncSinkWriter#retryForEntries` until
`http.sink.max-retries` is exhausted.
- Preserve legacy `http.sink.error.code.exclude` behavior by mapping it to
ignored response codes when the new option is not set.
- Persist retry attempt count in sink writer state, with restore
compatibility
for version 1 serialized state.
## Compatibility note
- Existing `HttpSinkRequestEntry` construction remains supported.
- Existing `http.sink.error.code.exclude` configurations continue to work.
- `HttpSinkWriterStateSerializer` is bumped from version 1 to version 2, with
version 1 deserialization compatibility.
- Default sink behavior remains success-only unless retry or ignored response
codes are configured.
## Verifying this change
Automated tests:
- `HttpSinkResponseClassifierTest`
- `HttpSinkWriterTest`
- `HttpSinkWriterStateSerializerTest`
- `HttpSinkConfigSerializationTest`
- `HttpDynamicTableSinkFactoryTest`
- `JavaNetSinkHttpClientTest`
- `HttpSinkConnectionTest`
Ran locally with JDK 17:
`mvn -pl flink-connector-http -DskipITs -Dskip.archunit=true
-Dspotless.check.skip=true -Dcheckstyle.skip=true
-Dtest=HttpSinkWriterTest,HttpSinkResponseClassifierTest,HttpSinkConfigSerializationTest,HttpSinkWriterStateSerializerTest,HttpDynamicTableSinkFactoryTest,HttpSinkConnectionTest,JavaNetSinkHttpClientTest
test`
Also ran:
`mvn -pl flink-connector-http -DskipTests compile`
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): **no**
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: **no**
- The serializers: **yes**, `HttpSinkWriterStateSerializer` is upgraded to
version 2 with version 1 restore compatibility
- The runtime per-record code paths (performance sensitive): **yes**, sink
response handling now classifies and retries retryable responses
- Anything that affects deployment or recovery: **yes**, sink writer state
now
stores retry attempt count
## Documentation
- Does this pull request introduce a new feature? **yes**
- If yes, how is the feature documented? Follow-up documentation work is
tracked
under the FLINK-40275 sink umbrella.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
