Danny Cranmer created FLINK-20629: ------------------------------------- Summary: [Kinesis][EFO] Migrate from DescribeStream to DescribeStreamSummary Key: FLINK-20629 URL: https://issues.apache.org/jira/browse/FLINK-20629 Project: Flink Issue Type: Improvement Components: Connectors / Kinesis Affects Versions: 1.12.0 Reporter: Danny Cranmer Fix For: 1.12.1
*Background* The Kinesis EFO connector invokes {{DescribeStream}} during startup to acquire the stream ARN. This call also includes the shard information and has a TPS of 10. A similar service exists, {{DescribeStreamSummary}} that has a TPS of 20 and a lighter response payload size. During startup sources with high parallelism compete to call this service (in {{LAZY}} mode), resulting in backoff and retry. Essentially the startup time can grow by 1s for every 10 parallelism, due to the 10 TPS. Migrating to {{DescribeStreamSummary}} will improve startup time. *Scope* Migrate call to {{DescribeStream}} to use {{DescribeStreamSummary}} instead. *Note* I have targeted {{1.12.1}}, let me know if we should instead target {{1.13}} -- This message was sent by Atlassian Jira (v8.3.4#803005)