Ravi Bhushan Ratnakar created FLINK-10536:
---------------------------------------------
Summary: Flink Kinesis Consumer leading to job failure due to
ProvisionedThroughputExceededException
Key: FLINK-10536
URL: https://issues.apache.org/jira/browse/FLINK-10536
Project: Flink
Issue Type: Improvement
Components: Kinesis Connector
Affects Versions: 1.5.2
Reporter: Ravi Bhushan Ratnakar
*Background:*
Flink Kinesis consumer, while consuming message from a shard, it retries for a
configured number of attempts and then after it throws
"ProvisionedThroughputExceededException" and which bubbles up to top level and
eventually which leads to failure of the running streaming job.
*Improvement:*
In some scenario it may be desirable that to skip some of the records by
skipping that "shardIterator" and move to next iterator in "ShardConsumer" and
log this as metrics. This can be configurable through some property. This way
those application which can tolerate to skip some messages can continue to keep
running.
If this idea seems useful, i would like to contribute on this. Please let me
know the process
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
