[ https://issues.apache.org/jira/browse/FLINK-24041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412322#comment-17412322 ]
Piotr Nowojski edited comment on FLINK-24041 at 9/9/21, 4:18 AM: ----------------------------------------------------------------- I had to revert this as 1702c9e and 16ab8b41af0 because of build failures. Looks like this PR hasn't been rebased and conflicted with another change in the master. https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=23814&view=logs&j=52b61abe-a3cc-5bde-cc35-1bbe89bb7df5&t=54421a62-0c80-5aad-3319-094ff69180bb {noformat} [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project flink-connector-base: Compilation failure [ERROR] /__w/2/s/flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/sink/writer/AsyncSinkWriterTest.java:[309,20] org.apache.flink.connector.base.sink.writer.AsyncSinkWriterTest.SinkInitContext is not abstract and does not override abstract method getRestoredCheckpointId() in org.apache.flink.api.connector.sink.Sink.InitContext {noformat} was (Author: pnowojski): I had to revert this as 1702c9e because of build failures. Looks like this PR hasn't been rebased and conflicted with another change in the master. https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=23814&view=logs&j=52b61abe-a3cc-5bde-cc35-1bbe89bb7df5&t=54421a62-0c80-5aad-3319-094ff69180bb {noformat} [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on project flink-connector-base: Compilation failure [ERROR] /__w/2/s/flink-connectors/flink-connector-base/src/test/java/org/apache/flink/connector/base/sink/writer/AsyncSinkWriterTest.java:[309,20] org.apache.flink.connector.base.sink.writer.AsyncSinkWriterTest.SinkInitContext is not abstract and does not override abstract method getRestoredCheckpointId() in org.apache.flink.api.connector.sink.Sink.InitContext {noformat} > [FLIP-171] Generic AsyncSinkBase > -------------------------------- > > Key: FLINK-24041 > URL: https://issues.apache.org/jira/browse/FLINK-24041 > Project: Flink > Issue Type: New Feature > Components: Connectors / Common > Reporter: Zichen Liu > Assignee: Zichen Liu > Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > h2. Motivation > Apache Flink has a rich connector ecosystem that can persist data in various > destinations. Flink natively supports Apache Kafka, Amazon Kinesis Data > Streams, Elasticsearch, HBase, and many more destinations. Additional > connectors are maintained in Apache Bahir or directly on GitHub. The basic > functionality of these sinks is quite similar. They batch events according to > user defined buffering hints, sign requests and send them to the respective > endpoint, retry unsuccessful or throttled requests, and participate in > checkpointing. They primarily just differ in the way they interface with the > destination. Yet, all the above-mentioned sinks are developed and maintained > independently. > We hence propose to create a sink that abstracts away this common > functionality into a generic sink. Adding support for a new destination then > just means creating a lightweight shim that only implements the specific > interfaces of the destination using a client that supports async requests. > Having a common abstraction will reduce the effort required to maintain all > these individual sinks. It will also make it much easier and faster to create > integrations with additional destinations. Moreover, improvements or bug > fixes to the core of the sink will benefit all implementations that are based > on it. > The design of the sink focusses on extensibility and a broad support of > destinations. The core of the sink is kept generic and free of any connector > specific dependencies. The sink is designed to participate in checkpointing > to provide at-least once semantics, but it is limited to destinations that > provide a client that supports async requests. > h2. References > More details to be found > https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink -- This message was sent by Atlassian Jira (v8.3.4#803005)