Hi all, we are struggling with RateLimitExceededExceptions with the Kinesis Producer. The Flink documentation claims that the Flink Producer overrides the RateLimit setting from Amazon's default of 150 to 100.
I am wondering whether we'd need 100/($sink_parallelism) in order for it to work correctly. Since the shard partitioner works on a provided key, every parallel flink sink may use all the shards, right? And since the different parallel Flink sinks cannot coordinate this, every parallel sink will try to saturate every shard, thereby overestimating the capacity by $sink_parallelism. Does anyone else have experience with or knowledge about this? Best regards, Urs -- Urs Schönenberger - urs.schoenenber...@tngtech.com TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring Geschäftsführer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Müller Sitz: Unterföhring * Amtsgericht München * HRB 135082
