Ying Xu created FLINK-14039:
-------------------------------
Summary: Flink Kinesis consumer: configurable per-shard
consumption rate when running in adaptive mode
Key: FLINK-14039
URL: https://issues.apache.org/jira/browse/FLINK-14039
Project: Flink
Issue Type: Improvement
Components: Connectors / Kinesis
Reporter: Ying Xu
Currently, Flink kinesis connector has a fixed
[2MB|https://github.com/apache/flink/blob/78748ea1aee8f9d0c0499180a2ef455490b32b24/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/internals/ShardConsumer.java#L59-L61]
target rate (per-shard) when running in adaptive rate mode. In specific
scenarios, it is desirable that users would want a different target rate. For
example, when two Kinesis consumers share a common stream, the user may want to
de-prioritize one stream such that it runs with a target rate < 2MB.
It is relatively straightforward to implement this feature – simply add a
per-shard target rate consumer config and has the default set to 2MB.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
