[jira] [Updated] (FLINK-7366) Upgrade kinesis producer library, kinesis client library, and AWS SDK in flink-connector-kinesis
[ https://issues.apache.org/jira/browse/FLINK-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bowen Li updated FLINK-7366: Description: We need to upgrade KPL and KCL to pick up the enhanced performance and stability for Flink to work better with Kinesis. Upgrading KPL is specially necessary, because the KPL version Flink uses is old, and doesn't have good retry and error handling logic. *KPL:* flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus problematic. It doesn't even have good retry logic, therefore Flink fails really frequently (about every 10 mins as we observed) when Flink writes too fast to Kinesis and receives RateLimitExceededException, Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, "*With the newer version of the KPL it uses the AWS C++ SDK which should offer additional retries.*" on Oct 2016. 0.12.5, the version we are upgrading to, is released in May 2017 and should have the enhanced retry logic. *KCL:* Upgrade KCL from 1.6.2 to 1.8.1 *AWS SDK* from 1.10.71 to 1.11.171 was: We need to upgrade KPL and KCL to pick up the enhanced performance and stability for Flink to work better with Kinesis. *KPL:* flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus problematic. It doesn't even have good retry logic, therefore Flink fails really frequently (about every 10 mins as we observed) when Flink writes too fast to Kinesis and receives RateLimitExceededException, Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, "*With the newer version of the KPL it uses the AWS C++ SDK which should offer additional retries.*" on Oct 2016. 0.12.5, the version we are upgrading to, is released in May 2017 and should have the enhanced retry logic. *KCL:* Upgrade KCL from 1.6.2 to 1.8.1 *AWS SDK* from 1.10.71 to 1.11.171 > Upgrade kinesis producer library, kinesis client library, and AWS SDK in > flink-connector-kinesis > > > Key: FLINK-7366 > URL: https://issues.apache.org/jira/browse/FLINK-7366 > Project: Flink > Issue Type: Bug > Components: Kinesis Connector >Affects Versions: 1.3.2 >Reporter: Bowen Li >Assignee: Bowen Li > Fix For: 1.4.0, 1.3.3 > > > We need to upgrade KPL and KCL to pick up the enhanced performance and > stability for Flink to work better with Kinesis. Upgrading KPL is specially > necessary, because the KPL version Flink uses is old, and doesn't have good > retry and error handling logic. > *KPL:* > flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which > is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus > problematic. It doesn't even have good retry logic, therefore Flink fails > really frequently (about every 10 mins as we observed) when Flink writes too > fast to Kinesis and receives RateLimitExceededException, > Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, > "*With the newer version of the KPL it uses the AWS C++ SDK which should > offer additional retries.*" on Oct 2016. 0.12.5, the version we are upgrading > to, is released in May 2017 and should have the enhanced retry logic. > *KCL:* > Upgrade KCL from 1.6.2 to 1.8.1 > *AWS SDK* > from 1.10.71 to 1.11.171 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (FLINK-7366) Upgrade kinesis producer library, kinesis client library, and AWS SDK in flink-connector-kinesis
[ https://issues.apache.org/jira/browse/FLINK-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bowen Li updated FLINK-7366: Issue Type: Bug (was: Improvement) > Upgrade kinesis producer library, kinesis client library, and AWS SDK in > flink-connector-kinesis > > > Key: FLINK-7366 > URL: https://issues.apache.org/jira/browse/FLINK-7366 > Project: Flink > Issue Type: Bug > Components: Kinesis Connector >Affects Versions: 1.3.2 >Reporter: Bowen Li >Assignee: Bowen Li > Fix For: 1.4.0, 1.3.3 > > > We need to upgrade KPL and KCL to pick up the enhanced performance and > stability for Flink to work better with Kinesis. > *KPL:* > flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which > is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus > problematic. It doesn't even have good retry logic, therefore Flink fails > really frequently (about every 10 mins as we observed) when Flink writes too > fast to Kinesis and receives RateLimitExceededException, > Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, > "*With the newer version of the KPL it uses the AWS C++ SDK which should > offer additional retries.*" on Oct 2016. 0.12.5, the version we are upgrading > to, is released in May 2017 and should have the enhanced retry logic. > *KCL:* > Upgrade KCL from 1.6.2 to 1.8.1 > *AWS SDK* > from 1.10.71 to 1.11.171 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (FLINK-7366) Upgrade kinesis producer library, kinesis client library, and AWS SDK in flink-connector-kinesis
[ https://issues.apache.org/jira/browse/FLINK-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bowen Li updated FLINK-7366: Description: We need to upgrade KPL and KCL to pick up the enhanced performance and stability for Flink to work better with Kinesis. *KPL:* flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus problematic. It doesn't even have good retry logic, therefore Flink fails really frequently (about every 10 mins as we observed) when Flink writes too fast to Kinesis and receives RateLimitExceededException, Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, "*With the newer version of the KPL it uses the AWS C++ SDK which should offer additional retries.*" on Oct 2016. 0.12.5, the version we are upgrading to, is released in May 2017 and should have the enhanced retry logic. *KCL:* Upgrade KCL from 1.6.2 to 1.8.1 *AWS SDK* from 1.10.71 to 1.11.171 was: *KPL:* flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus problematic. It doesn't even have good retry logic, therefore Flink fails really frequently (about every 10 mins as we observed) when Flink writes too fast to Kinesis and receives RateLimitExceededException, Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, "*With the newer version of the KPL it uses the AWS C++ SDK which should offer additional retries.*" on Oct 2016. Upgrade it to 0.12.5 released in May 2017 to take advantages of all the new features and improvements. *KCL:* Upgrade KCL from 1.6.2 to 1.8.1 *AWS SDK* from 1.10.71 to 1.11.171 > Upgrade kinesis producer library, kinesis client library, and AWS SDK in > flink-connector-kinesis > > > Key: FLINK-7366 > URL: https://issues.apache.org/jira/browse/FLINK-7366 > Project: Flink > Issue Type: Improvement > Components: Kinesis Connector >Affects Versions: 1.3.2 >Reporter: Bowen Li >Assignee: Bowen Li > Fix For: 1.4.0, 1.3.3 > > > We need to upgrade KPL and KCL to pick up the enhanced performance and > stability for Flink to work better with Kinesis. > *KPL:* > flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which > is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus > problematic. It doesn't even have good retry logic, therefore Flink fails > really frequently (about every 10 mins as we observed) when Flink writes too > fast to Kinesis and receives RateLimitExceededException, > Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, > "*With the newer version of the KPL it uses the AWS C++ SDK which should > offer additional retries.*" on Oct 2016. 0.12.5, the version we are upgrading > to, is released in May 2017 and should have the enhanced retry logic. > *KCL:* > Upgrade KCL from 1.6.2 to 1.8.1 > *AWS SDK* > from 1.10.71 to 1.11.171 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (FLINK-7366) Upgrade kinesis producer library, kinesis client library, and AWS SDK in flink-connector-kinesis
[ https://issues.apache.org/jira/browse/FLINK-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bowen Li updated FLINK-7366: Description: *KPL:* flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus problematic. It doesn't even have good retry logic, therefore Flink fails really frequently (about every 10 mins as we observed) when Flink writes too fast to Kinesis and receives RateLimitExceededException, Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, "*With the newer version of the KPL it uses the AWS C++ SDK which should offer additional retries.*" on Oct 2016. Upgrade it to 0.12.5 released in May 2017 to take advantages of all the new features and improvements. *KCL:* Upgrade KCL from 1.6.2 to 1.8.1 *AWS SDK* from 1.10.71 to 1.11.171 was: *KPL: * flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus problematic. It doesn't even have good retry logic, therefore Flink fails really frequently (about every 10 mins as we observed) when Flink writes too fast to Kinesis and receives RateLimitExceededException, Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, "*With the newer version of the KPL it uses the AWS C++ SDK which should offer additional retries.*" on Oct 2016. Upgrade it to 0.12.5 released in May 2017 to take advantages of all the new features and improvements. *KCL:* Upgrade KCL from 1.6.2 to 1.8.1 *AWS SDK* from 1.10.71 to 1.11.171 > Upgrade kinesis producer library, kinesis client library, and AWS SDK in > flink-connector-kinesis > > > Key: FLINK-7366 > URL: https://issues.apache.org/jira/browse/FLINK-7366 > Project: Flink > Issue Type: Improvement > Components: Kinesis Connector >Affects Versions: 1.3.2 >Reporter: Bowen Li >Assignee: Bowen Li > Fix For: 1.4.0, 1.3.3 > > > *KPL:* > flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which > is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus > problematic. It doesn't even have good retry logic, therefore Flink fails > really frequently (about every 10 mins as we observed) when Flink writes too > fast to Kinesis and receives RateLimitExceededException, > Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, > "*With the newer version of the KPL it uses the AWS C++ SDK which should > offer additional retries.*" on Oct 2016. > Upgrade it to 0.12.5 released in May 2017 to take advantages of all the new > features and improvements. > *KCL:* > Upgrade KCL from 1.6.2 to 1.8.1 > *AWS SDK* > from 1.10.71 to 1.11.171 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (FLINK-7366) Upgrade kinesis producer library, kinesis client library, and AWS SDK in flink-connector-kinesis
[ https://issues.apache.org/jira/browse/FLINK-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bowen Li updated FLINK-7366: Description: *KPL: * flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus problematic. It doesn't even have good retry logic, therefore Flink fails really frequently (about every 10 mins as we observed) when Flink writes too fast to Kinesis and receives RateLimitExceededException, Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, "*With the newer version of the KPL it uses the AWS C++ SDK which should offer additional retries.*" on Oct 2016. Upgrade it to 0.12.5 released in May 2017 to take advantages of all the new features and improvements. *KCL:* Upgrade KCL from 1.6.2 to 1.8.1 *AWS SDK* from 1.10.71 to 1.11.171 was: # KPL: flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus problematic. It doesn't even have good retry logic, therefore Flink fails really frequently (about every 10 mins as we observed) when Flink writes too fast to Kinesis and receives RateLimitExceededException, Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, "*With the newer version of the KPL it uses the AWS C++ SDK which should offer additional retries.*" on Oct 2016. Upgrade it to 0.12.5 released in May 2017 to take advantages of all the new features and improvements. # KCL: Upgrade KCL from 1.6.2 to 1.8.1 # AWS SDK from 1.10.71 to 1.11.171 > Upgrade kinesis producer library, kinesis client library, and AWS SDK in > flink-connector-kinesis > > > Key: FLINK-7366 > URL: https://issues.apache.org/jira/browse/FLINK-7366 > Project: Flink > Issue Type: Improvement > Components: Kinesis Connector >Affects Versions: 1.3.2 >Reporter: Bowen Li >Assignee: Bowen Li > Fix For: 1.4.0, 1.3.3 > > > *KPL: * > flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which > is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus > problematic. It doesn't even have good retry logic, therefore Flink fails > really frequently (about every 10 mins as we observed) when Flink writes too > fast to Kinesis and receives RateLimitExceededException, > Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, > "*With the newer version of the KPL it uses the AWS C++ SDK which should > offer additional retries.*" on Oct 2016. > Upgrade it to 0.12.5 released in May 2017 to take advantages of all the new > features and improvements. > *KCL:* > Upgrade KCL from 1.6.2 to 1.8.1 > *AWS SDK* > from 1.10.71 to 1.11.171 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (FLINK-7366) Upgrade kinesis producer library, kinesis client library, and AWS SDK in flink-connector-kinesis
[ https://issues.apache.org/jira/browse/FLINK-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bowen Li updated FLINK-7366: Description: # KPL: flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus problematic. It doesn't even have good retry logic, therefore Flink fails really frequently (about every 10 mins as we observed) when Flink writes too fast to Kinesis and receives RateLimitExceededException, Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, "*With the newer version of the KPL it uses the AWS C++ SDK which should offer additional retries.*" on Oct 2016. Upgrade it to 0.12.5 released in May 2017 to take advantages of all the new features and improvements. # KCL: Upgrade KCL from 1.6.2 to 1.8.1 # AWS SDK from 1.10.71 to 1.11.171 was: flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus problematic. It doesn't even have good retry logic, therefore Flink fails really frequently (about every 10 mins as we observed) when Flink writes too fast to Kinesis and receives RateLimitExceededException, Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, "*With the newer version of the KPL it uses the AWS C++ SDK which should offer additional retries.*" on Oct 2016. Upgrade it to 0.12.5 released in May 2017 to take advantages of all the new features and improvements. > Upgrade kinesis producer library, kinesis client library, and AWS SDK in > flink-connector-kinesis > > > Key: FLINK-7366 > URL: https://issues.apache.org/jira/browse/FLINK-7366 > Project: Flink > Issue Type: Improvement > Components: Kinesis Connector >Affects Versions: 1.3.2 >Reporter: Bowen Li >Assignee: Bowen Li > Fix For: 1.4.0, 1.3.3 > > > # KPL: > flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which > is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus > problematic. It doesn't even have good retry logic, therefore Flink fails > really frequently (about every 10 mins as we observed) when Flink writes too > fast to Kinesis and receives RateLimitExceededException, > Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, > "*With the newer version of the KPL it uses the AWS C++ SDK which should > offer additional retries.*" on Oct 2016. > Upgrade it to 0.12.5 released in May 2017 to take advantages of all the new > features and improvements. > # KCL: > Upgrade KCL from 1.6.2 to 1.8.1 > # AWS SDK > from 1.10.71 to 1.11.171 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (FLINK-7366) Upgrade kinesis producer library, kinesis client library, and AWS SDK in flink-connector-kinesis
[ https://issues.apache.org/jira/browse/FLINK-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bowen Li updated FLINK-7366: Summary: Upgrade kinesis producer library, kinesis client library, and AWS SDK in flink-connector-kinesis (was: Upgrade kinesis-producer-library in flink-connector-kinesis from 0.10.2 to 0.12.5) > Upgrade kinesis producer library, kinesis client library, and AWS SDK in > flink-connector-kinesis > > > Key: FLINK-7366 > URL: https://issues.apache.org/jira/browse/FLINK-7366 > Project: Flink > Issue Type: Improvement > Components: Kinesis Connector >Affects Versions: 1.3.2 >Reporter: Bowen Li >Assignee: Bowen Li > Fix For: 1.4.0, 1.3.3 > > > flink-connector-kinesis currently uses kinesis-producer-library 0.10.2, which > is released in Nov 2015 by AWS. It's old. It's the fourth release, and thus > problematic. It doesn't even have good retry logic, therefore Flink fails > really frequently (about every 10 mins as we observed) when Flink writes too > fast to Kinesis and receives RateLimitExceededException, > Quotes from https://github.com/awslabs/amazon-kinesis-producer/issues/56, > "*With the newer version of the KPL it uses the AWS C++ SDK which should > offer additional retries.*" on Oct 2016. > Upgrade it to 0.12.5 released in May 2017 to take advantages of all the new > features and improvements. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
