It will be good to be able to use the ListShards API. Are there any
concerns bumping up the AWS SDK dependency? I see it was last done in


On Wed, Feb 28, 2018 at 10:38 PM, Kailash Dayanand <>

> Based on the discussion at here
> <>,
> I want to propose using the latest ListShards API instead of the
> DescribeStreams on AWS to overcome the rate limits currently imposed on
> DescribeStream. The new List Shards have a much higher rate limits (a
> limit of 100 transactions per second per data stream link
> <>).
> This was recently introduced in the aws-sdk-java release of 1.11.272
> <>. I propose
> bumping up the aws-sdk-java used in flink-kinesis connector and replace the
> DescribeStream calls with ListShards in the KinesisProxy class here
> <>
>  allowing
> for faster shard discovery rate.
> Thanks
> Kailash

Reply via email to