[
https://issues.apache.org/jira/browse/FLINK-26890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17628394#comment-17628394
]
Danny Cranmer edited comment on FLINK-26890 at 11/4/22 9:30 AM:
----------------------------------------------------------------
Merged:
-
[{{68adbcc}}|https://github.com/apache/flink/commit/68adbcc8823a1f42e1911ddfd8ab5d4ea111043b]
into master
-
[{{f410434}}|https://github.com/apache/flink/commit/f410434e83558191bed2e9f0874f0634ca56dfa1]
into release-1.16
was (Author: dannycranmer):
Merged commit
[{{68adbcc}}|https://github.com/apache/flink/commit/68adbcc8823a1f42e1911ddfd8ab5d4ea111043b]
into master
> DynamoDB consumer error consuming partitions close to retention
> ---------------------------------------------------------------
>
> Key: FLINK-26890
> URL: https://issues.apache.org/jira/browse/FLINK-26890
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kinesis
> Affects Versions: 1.16.0, 1.15.2
> Reporter: Danny Cranmer
> Assignee: Elphas Toringepi
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.17.0, 1.15.3, 1.16.1
>
>
> *Background*
> The Amazon Kinesis Data Streams consumer supports consuming from Amazon
> DynamoDB via the [DynamoDB Streams Kinesis
> Adapter|https://github.com/awslabs/dynamodb-streams-kinesis-adapter].
> *Problem*
> We have seen instances of consumer throwing {{ResouceNotFoundException}} when
> attempting to invoke {{GetShardIterator}}.
> {code}
> com.amazonaws.services.kinesis.model.ResourceNotFoundException: Requested
> resource not found: Shard does not exist
> {code}
> According to the DynamoDB team, the {{DescribeStream}} call may return shard
> IDs that are no longer valid, and this exception needs to be handled by the
> client.
> *Solution*
> Modify the DynamoDB consumer to treat {{ResourceNotFoundException}} as a
> shard closed signal.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
