[
https://issues.apache.org/jira/browse/FLINK-36296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18044352#comment-18044352
]
Ferenc Csaky edited comment on FLINK-36296 at 1/22/26 8:09 PM:
---------------------------------------------------------------
[{{68acdbd}}|https://github.com/apache/flink-connector-aws/commit/68acdbd69b9ba5beb00e26abf277b6ec5c660e92]
on main
[e622064|https://github.com/apache/flink-connector-aws/commit/e622064f4a1a33e996581e9bebfc44133eb74bc8]
on v6.0
was (Author: JIRAUSER306586):
[{{68acdbd}}|https://github.com/apache/flink-connector-aws/commit/68acdbd69b9ba5beb00e26abf277b6ec5c660e92]
on main
> Add incremental shard discovery for DDB Streams source
> ------------------------------------------------------
>
> Key: FLINK-36296
> URL: https://issues.apache.org/jira/browse/FLINK-36296
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / DynamoDB
> Reporter: Abhi Gupta
> Assignee: Abhi Gupta
> Priority: Major
> Labels: pull-request-available
> Fix For: aws-connector-6.0.0
>
>
> DDB Streams source does a full shard discovery every minute, on large
> streams, this discovery can take more than 20 minutes to discover all shards,
> we can optimize this to do an incremental shard discovery periodically while
> also doing periodic shard discovery. This makes having the children shards
> faster and reduces the millisBehindLatest
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
