[
https://issues.apache.org/jira/browse/FLINK-24228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-24228:
-----------------------------------
Labels: pull-request-available stale-assigned (was: pull-request-available)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issue is assigned but has not
received an update in 30 days, so it has been labeled "stale-assigned".
If you are still working on the issue, please remove the label and add a
comment updating the community on your progress. If this issue is waiting on
feedback, please consider this a reminder to the committer/reviewer. Flink is a
very active project, and so we appreciate your patience.
If you are no longer working on the issue, please unassign yourself so someone
else may work on it.
> [FLIP-171] Firehose implementation of Async Sink
> ------------------------------------------------
>
> Key: FLINK-24228
> URL: https://issues.apache.org/jira/browse/FLINK-24228
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Common
> Reporter: Zichen Liu
> Assignee: Zichen Liu
> Priority: Major
> Labels: pull-request-available, stale-assigned
> Fix For: 1.15.0
>
>
> h2. Motivation
> *User stories:*
> As a Flink user, I’d like to use Kinesis Firehose as sink for my data
> pipeline.
> *Scope:*
> * Implement an asynchronous sink for Kinesis Firehose by inheriting the
> AsyncSinkBase class. The implementation can for now reside in its own module
> in flink-connectors. The module and package name can be anything reasonable
> e.g. {{flink-connector-aws-kinesis}} for the module name and
> {{org.apache.flink.connector.aws.kinesis}} for the package name.
> * The implementation must use [the Kinesis Java
> Client|https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/kinesis/AmazonKinesisClient.html].
> * The implementation must allow users to configure the Kinesis Client, with
> reasonable default settings.
> * Implement an asynchornous sink writer for Firehose by extending the
> AsyncSinkWriter. The implementation must deal with failed requests and retry
> them using the {{requeueFailedRequestEntry}} method. If possible, the
> implementation should batch multiple requests (PutRecordsRequestEntry
> objects) to Firehose for increased throughput. The implemented Sink Writer
> will be used by the Sink class that will be created as part of this story.
> * Unit/Integration testing. Use Kinesalite (in-memory Kinesis simulation).
> We already use this in {{KinesisTableApiITCase}}.
> * Java / code-level docs.
> * End to end testing: add tests that hits a real AWS instance. (How to best
> donate resources to the Flink project to allow this to happen?)
> h2. References
> More details to be found
> [https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
