[
https://issues.apache.org/jira/browse/FLINK-26890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Huang Xingbo updated FLINK-26890:
---------------------------------
Fix Version/s: 1.14.6
(was: 1.14.5)
> [Kinesis][Consumer] DynamoDB consumer error consuming partitions close to
> retention
> -----------------------------------------------------------------------------------
>
> Key: FLINK-26890
> URL: https://issues.apache.org/jira/browse/FLINK-26890
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kinesis
> Affects Versions: 1.15.0
> Reporter: Danny Cranmer
> Priority: Major
> Fix For: 1.16.0, 1.13.7, 1.14.6
>
>
> *Background*
> The Amazon Kinesis Data Streams consumer supports consuming from Amazon
> DynamoDB via the [DynamoDB Streams Kinesis
> Adapter|https://github.com/awslabs/dynamodb-streams-kinesis-adapter].
> *Problem*
> We have seen instances of consumer throwing {{ResouceNotFoundException}} when
> attempting to invoke {{GetShardIterator}}.
> {code}
> com.amazonaws.services.kinesis.model.ResourceNotFoundException: Requested
> resource not found: Shard does not exist
> {code}
> According to the DynamoDB team, the {{DescribeStream}} call may return shard
> IDs that are no longer valid, and this exception needs to be handled by the
> client.
> *Solution*
> Modify the DynamoDB consumer to treat {{ResourceNotFoundException}} as a
> shard closed signal.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
