[
https://issues.apache.org/jira/browse/FLINK-25731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zichen Liu updated FLINK-25731:
-------------------------------
Description:
This consumer based on the Kinesis Consumer Library (KCL) has been deprecated,
a Jira to create an equivalent consumer based on AWS SDK for Java 2.x has been
raised in FLINK-25729 and planned for Flink 1.16.
This producer based on the Kinesis Producer Library KPL has been superseded.
The new sink can be found in the module
flink-connectors/flink-connector-aws-kinesis-data-streams and package
org.apache.flink.connector.kinesis.sink.KinesisDataStreamsSink. It is based on
the AWS SDK for Java 2.x. The work to replace this sink was carried out in
FLINK-24227.
was:
h2. Motivation
*User stories:*
As a Flink user, I’d like to use DynamoDB as sink for my data pipeline.
*Scope:*
* Implement an asynchronous sink for DynamoDB by inheriting the AsyncSinkBase
class. The implementation can for now reside in its own module in
flink-connectors.
* Implement an asynchornous sink writer for DynamoDB by extending the
AsyncSinkWriter. The implementation must deal with failed requests and retry
them using the {{requeueFailedRequestEntry}} method. If possible, the
implementation should batch multiple requests (PutRecordsRequestEntry objects)
to Firehose for increased throughput. The implemented Sink Writer will be used
by the Sink class that will be created as part of this story.
* Java / code-level docs.
* End to end testing: add tests that hits a real AWS instance. (How to best
donate resources to the Flink project to allow this to happen?)
h2. References
More details to be found
[https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink]
> Mark FlinkKinesisProducer/FlinkKinesisConsumer as deprecated
> ------------------------------------------------------------
>
> Key: FLINK-25731
> URL: https://issues.apache.org/jira/browse/FLINK-25731
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Common
> Reporter: Zichen Liu
> Assignee: Zichen Liu
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
> This consumer based on the Kinesis Consumer Library (KCL) has been
> deprecated, a Jira to create an equivalent consumer based on AWS SDK for Java
> 2.x has been raised in FLINK-25729 and planned for Flink 1.16.
> This producer based on the Kinesis Producer Library KPL has been superseded.
> The new sink can be found in the module
> flink-connectors/flink-connector-aws-kinesis-data-streams and package
> org.apache.flink.connector.kinesis.sink.KinesisDataStreamsSink. It is based
> on the AWS SDK for Java 2.x. The work to replace this sink was carried out in
> FLINK-24227.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
