mosche commented on code in PR #23540:
URL: https://github.com/apache/beam/pull/23540#discussion_r1145998464
##########
sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisIO.java:
##########
@@ -147,6 +150,42 @@
* Read#withCustomRateLimitPolicy(RateLimitPolicyFactory)}. This requires
implementing {@link
* RateLimitPolicy} with a corresponding {@link RateLimitPolicyFactory}.
*
+ * <h4>Enhanced Fan-Out</h4>
+ *
+ * Kinesis IO supports Consumers with Dedicated Throughput (Enhanced Fan-Out,
EFO). This type of
+ * consumer doesn't have to contend with other consumers that are receiving
data from the stream.
+ *
+ * <p>More details can be found here: <a
+ *
href="https://docs.aws.amazon.com/streams/latest/dev/enhanced-consumers.html";>Consumers
with
+ * Dedicated Throughput</a>
+ *
+ * <p>EFO can be enabled for one or more {@link KinesisIO.Read} instances via
pipeline options:
Review Comment:
> Any tips? If there is no way to have IDs in Read and
AutoValue_KinesisIO_Read both hard-coded to different values, what should be
the way forward?
Unfortunately not, I'm not aware of any possible way to that. The mechanism
of `@AutoValue` is fairly limited there :(
I'm more and more realizing how bad the behavior of the Flink runner is :/
Adding new configuration options to Read specs is a common task. I think we
should drop our requirement to keep Java serialization of KinesisSource /
KinesisIO.Read backwards compatible and consider this an issue of the Flink
runner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
