[ https://issues.apache.org/jira/browse/KAFKA-5828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16373773#comment-16373773 ]
Ewen Cheslack-Postava commented on KAFKA-5828: ---------------------------------------------- I don't think this is best solved at the Connect level. Could you just use [quotas|http://kafka.apache.org/documentation.html#design_quotas] to rate limit the connector? > Allow explicitly setting polling interval for Kafka connectors > -------------------------------------------------------------- > > Key: KAFKA-5828 > URL: https://issues.apache.org/jira/browse/KAFKA-5828 > Project: Kafka > Issue Type: Improvement > Components: KafkaConnect > Reporter: Behrang Saeedzadeh > Priority: Major > > I have a Kafka cluster deployed in our internal data center and a Kafka > Connect server deployed in AWS that gets data records from a topic on the > Kafka cluster and writes them into a Kinesis stream. > We want to ensure that our Kafka connect server does not saturate the > available bandwidth between our internal data center and AWS. > Using {{max.partition.fetch.bytes}} we can limit the upper bound of data that > can be fetched in each poll call. If we can also configure the poll interval, > then we can limit how much data is transferred per partition per second. -- This message was sent by Atlassian JIRA (v7.6.3#76005)