[
https://issues.apache.org/jira/browse/FLINK-19898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tzu-Li (Gordon) Tai closed FLINK-19898.
---------------------------------------
Assignee: Danny Cranmer
Resolution: Fixed
flink/master: 9d6309432108b446e59eb3c07142115b3af8b1a6
> [Kinesis][EFO] Ignore ReadTimeoutException from SubcribeToShard retry policy
> ----------------------------------------------------------------------------
>
> Key: FLINK-19898
> URL: https://issues.apache.org/jira/browse/FLINK-19898
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kinesis
> Reporter: Danny Cranmer
> Assignee: Danny Cranmer
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0
>
>
> *Background*
> The Flink Kinesis EFO consumer has a {{SubscribeToShard}} retry policy which
> will terminate the job after a given number of subsequent attempt failures.
> In high backpressure scenarios the Netty HTTP Client throws a
> {{ReadTimeoutException}} when the consumer takes longer than 30s to process a
> batch. If this happens (by default) 10 times in a row, the job will
> terminate. There is no need to terminate in this condition, and the restart
> results in the job falling further behind.
> *Scope*
> Exclude the {{ReadTimeoutException}} from the {{SubscribeToShard}} retry
> policy, such that that connector will gracefully reconnect once the consumer
> has processed the queued records.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
