[ 
https://issues.apache.org/jira/browse/FLINK-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bowen Li updated FLINK-7366:
----------------------------
    Description: 
flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which 
is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus 
problematic. It doesn't even have good retry logic, therefore Flink fails 
really frequently (about every 10 mins as we observed) when Flink writes too 
fast to Kinesis and receives RateLimitExceededException, 

Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, 
"*With the newer version of the KPL it uses the AWS C++ SDK which should offer 
additional retries.*" on Oct 2016.

Upgrade it to 0.12.5 released in May 2017 to take advantages of all the new 
features and improvements.

  was:
flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which 
is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus 
problematic. It doesn't even have good retry logic, therefore Flink fails 
really frequently (about every 10 mins as we observed) when Flink writes too 
fast to Kinesis and receives RateLimitExceededException, 

Upgrade it to 0.12.5 released in May 2017 to take advantages of all the new 
features and improvements.


> Upgrade kinesis-producer-library in flink-connector-kinesis from 0.10.2 to 
> 0.12.5
> ---------------------------------------------------------------------------------
>
>                 Key: FLINK-7366
>                 URL: https://issues.apache.org/jira/browse/FLINK-7366
>             Project: Flink
>          Issue Type: Improvement
>          Components: Kinesis Connector
>    Affects Versions: 1.3.2
>            Reporter: Bowen Li
>            Assignee: Bowen Li
>             Fix For: 1.4.0, 1.3.3
>
>
> flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which 
> is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus 
> problematic. It doesn't even have good retry logic, therefore Flink fails 
> really frequently (about every 10 mins as we observed) when Flink writes too 
> fast to Kinesis and receives RateLimitExceededException, 
> Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, 
> "*With the newer version of the KPL it uses the AWS C++ SDK which should 
> offer additional retries.*" on Oct 2016.
> Upgrade it to 0.12.5 released in May 2017 to take advantages of all the new 
> features and improvements.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to