mosche commented on issue #23570:
URL: https://github.com/apache/beam/issues/23570#issuecomment-1403667655
The x-lang writer, unfortunately, is rather broken from a design perspective
and will be barely usable for anything beyond testing. It cannot use more than
a single Kinesis shard.
| WriteToKinesis | KinesisIO.Write (v1) | KinesisIO.Write
(v2)
| Comments
|
|---------------------------------|------------------------|-----------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| stream_name | withStreamName | withStreamName
|
|
| aws_access_key / aws_secret_key | withAWSClientsProvider |
withClientConfiguration
| Use of static
credentials is a bad practice, use the [default credentials provider
chain](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials.html)
instead |
| region | withAWSClientsProvider |
withClientConfiguration
|
|
| partition_key | ~~withPartitionKey~~ | Configure a
random partitioner, e.g.
KinesisPartitioner</br>.explicitRandomPartitioner(shards)
| Configuring a single hardcoded partition key is totally
broken. That way records will only ever be published to a single shard, no
matter how many shards exist. |
| service_endpoint | withAWSClientsProvider |
withClientConfiguration
|
|
| verify_certificate | withAWSClientsProvider | can't be
disabled
|
|
| producer_properties | withProducerProperties | not using KPL,
related config options are withBatchMaxRecords, withBatchMaxBytes,
withConcurrentRequests, withRecordAggregationDisabled |
|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
