Danny Cranmer created FLINK-19898:
-------------------------------------
Summary: [Kinesis][EFO] Ignore ReadTimeoutException from
SubcribeToShard retry policy
Key: FLINK-19898
URL: https://issues.apache.org/jira/browse/FLINK-19898
Project: Flink
Issue Type: Improvement
Components: Connectors / Kinesis
Reporter: Danny Cranmer
Fix For: 1.12.0
*Background*
The Flink Kinesis EFO consumer has a {{SubscribeToShard}} retry policy which
will terminate the job after a given number of subsequent attempt failures. In
high backpressure scenarios the Netty HTTP Client throws a
{{ReadTimeoutException}} when the consumer takes longer than 30s to process a
batch. If this happens (by default) 10 times in a row, the job will terminate.
There is no need to terminate in this condition, and the restart results in the
job falling further behind.
*Scope*
Exclude the {{ReadTimeoutException}} from the {{SubscribeToShard}} retry
policy, such that that connector will gracefully reconnect once the consumer
has processed the queued records.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
