Brad Atcheson created FLINK-34023:
-------------------------------------
Summary: Expose Kinesis client retry config in sink
Key: FLINK-34023
URL: https://issues.apache.org/jira/browse/FLINK-34023
Project: Flink
Issue Type: Improvement
Components: Connectors / Kinesis
Reporter: Brad Atcheson
The consumer side exposes client retry configuration like
[flink.shard.getrecords.maxretries|https://nightlies.apache.org/flink/flink-docs-stable/api/java/org/apache/flink/streaming/connectors/kinesis/config/ConsumerConfigConstants.html#SHARD_GETRECORDS_RETRIES]
but the producer side lacks similar config for PutRecords.
The KinesisStreamsSinkWriter constructor calls
{code}
this.httpClient = AWSGeneralUtil.createAsyncHttpClient(kinesisClientProperties);
this.kinesisClient = buildClient(kinesisClientProperties, this.httpClient);
{code}
But those methods only refer to these values (aside from endpoint/region/creds)
in the kinesisClientProperties:
* aws.http-client.max-concurrency
* aws.http-client.read-timeout
* aws.trust.all.certificates
* aws.http.protocol.version
Without control over retry, users can observe exceptions like {code}Request
attempt 2 failure: Unable to execute HTTP request: connection timed out after
2000 ms: kinesis.us-west-2.amazonaws.com{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
