[
https://issues.apache.org/jira/browse/FLINK-24041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412322#comment-17412322
]
Piotr Nowojski edited comment on FLINK-24041 at 9/9/21, 3:54 AM:
-----------------------------------------------------------------
I had to revert this as 1702c9e because of build failures. Looks like this PR
hasn't been rebased and conflicted with another change in the master.
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=23814&view=logs&j=52b61abe-a3cc-5bde-cc35-1bbe89bb7df5&t=54421a62-0c80-5aad-3319-094ff69180bb
was (Author: pnowojski):
I had to revert this as 1702c9e because of build failures. Looks like this PR
hasn't been rebased and conflicted with another change in the master.
> [FLIP-171] Generic AsyncSinkBase
> --------------------------------
>
> Key: FLINK-24041
> URL: https://issues.apache.org/jira/browse/FLINK-24041
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / Common
> Reporter: Zichen Liu
> Assignee: Zichen Liu
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.15.0
>
>
> h2. Motivation
> Apache Flink has a rich connector ecosystem that can persist data in various
> destinations. Flink natively supports Apache Kafka, Amazon Kinesis Data
> Streams, Elasticsearch, HBase, and many more destinations. Additional
> connectors are maintained in Apache Bahir or directly on GitHub. The basic
> functionality of these sinks is quite similar. They batch events according to
> user defined buffering hints, sign requests and send them to the respective
> endpoint, retry unsuccessful or throttled requests, and participate in
> checkpointing. They primarily just differ in the way they interface with the
> destination. Yet, all the above-mentioned sinks are developed and maintained
> independently.
> We hence propose to create a sink that abstracts away this common
> functionality into a generic sink. Adding support for a new destination then
> just means creating a lightweight shim that only implements the specific
> interfaces of the destination using a client that supports async requests.
> Having a common abstraction will reduce the effort required to maintain all
> these individual sinks. It will also make it much easier and faster to create
> integrations with additional destinations. Moreover, improvements or bug
> fixes to the core of the sink will benefit all implementations that are based
> on it.
> The design of the sink focusses on extensibility and a broad support of
> destinations. The core of the sink is kept generic and free of any connector
> specific dependencies. The sink is designed to participate in checkpointing
> to provide at-least once semantics, but it is limited to destinations that
> provide a client that supports async requests.
> h2. References
> More details to be found
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
