infoverload commented on code in PR #517: URL: https://github.com/apache/flink-web/pull/517#discussion_r855137972
########## _posts/2022-03-16-async-sink-base.md: ########## @@ -0,0 +1,160 @@ +--- +layout: post +title: "The Generic Asynchronous Base Sink" +date: 2022-04-05 16:00:00 +authors: +- CrynetLogistics: + name: "Zichen Liu" +excerpt: An overview of the new AsyncBaseSink and how to use it for building your own concrete sink +--- + +Flink sinks in general are quite similar. They batch records according to user defined buffering hints, sign requests, write them to the destination, retry unsuccessful or throttled requests, and participate in checkpointing. Review Comment: ```suggestion Flink sinks share a lot of similar behavior. All sinks batch records according to user-defined buffering hints, sign requests, write them to the destination, retry unsuccessful or throttled requests, and participate in checkpointing. ``` ########## _posts/2022-03-16-async-sink-base.md: ########## @@ -0,0 +1,160 @@ +--- +layout: post +title: "The Generic Asynchronous Base Sink" +date: 2022-04-05 16:00:00 +authors: +- CrynetLogistics: + name: "Zichen Liu" +excerpt: An overview of the new AsyncBaseSink and how to use it for building your own concrete sink +--- + +Flink sinks in general are quite similar. They batch records according to user defined buffering hints, sign requests, write them to the destination, retry unsuccessful or throttled requests, and participate in checkpointing. + +New in [Flink 1.15](https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink) is the `AsyncSinkBase`, an abstract sink with a number of common functionalities extracted. Adding support for a new destination now only requires a lightweight shim that implements the specific interfaces of the destination using a client that supports async requests. Review Comment: ```suggestion This is why for [Flink 1.15](https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink) we have decided to create the `AsyncSinkBase`, an abstract sink with a number of common functionalities extracted. Adding support for a new destination now only requires a lightweight shim that implements the specific interfaces of the destination using a client that supports async requests. ``` ########## _posts/2022-03-16-async-sink-base.md: ########## @@ -0,0 +1,160 @@ +--- +layout: post +title: "The Generic Asynchronous Base Sink" +date: 2022-04-05 16:00:00 +authors: +- CrynetLogistics: + name: "Zichen Liu" +excerpt: An overview of the new AsyncBaseSink and how to use it for building your own concrete sink +--- + +Flink sinks in general are quite similar. They batch records according to user defined buffering hints, sign requests, write them to the destination, retry unsuccessful or throttled requests, and participate in checkpointing. + +New in [Flink 1.15](https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink) is the `AsyncSinkBase`, an abstract sink with a number of common functionalities extracted. Adding support for a new destination now only requires a lightweight shim that implements the specific interfaces of the destination using a client that supports async requests. + +This common abstraction will reduce the effort required to maintain individual sinks that extend from this abstract sink, with bugfixes and improvements to the sink core benefiting all implementations that extend it. + +**Note** The sink is designed to participate in checkpointing to provide at-least once semantics. + +**Attention** The sink can work directly with destinations that provide a client that supports asynchronous requests. Alternatively, concrete sink implementers may manage their own thread pool with a synchronous client. + +The design of the sink focuses on extensibility and a broad support of destinations. The core of the sink is kept generic and free of any connector-specific dependencies. Review Comment: ```suggestion This common abstraction will reduce the effort required to maintain individual sinks that extend from this abstract sink, with bugfixes and improvements to the sink core benefiting all implementations that extend it. The design of AsyncSyncBase focuses on extensibility and a broad support of destinations. The core of the sink is kept generic and free of any connector-specific dependencies. The sink base is designed to participate in checkpointing to provide at-least-once semantics and can work directly with destinations that provide a client that supports asynchronous requests. Alternatively, concrete sink implementers may manage their own thread pool with a synchronous client. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
