Re: [DISCUSS] FLIP-171: Async Sink

2021-08-20 Thread Hausmann, Steffen
ss you can confirm the sender and know the content is safe. Sure, thanks for the pointers. Cheers, Till On Fri, Jul 16, 2021 at 6:19 PM Hausmann, Steffen wrote: > Hi Till, > > You are right, I’ve left out some implementation details, which have

Re: [DISCUSS] FLIP-177: Extend Sink API

2021-07-30 Thread Hausmann, Steffen
> > > > It's a bit different. Let's take kinesis as an example. The sink > > collects > > > > 500 elements and puts them in a single request (500 is max number of > > > > records per batch request). Now it sends the request ou

Re: [DISCUSS] FLIP-177: Extend Sink API

2021-07-22 Thread Hausmann, Steffen
implementation internally (similarly to AsyncIO). There is an example for option 1 in [2]; I think the idea was to additionally specify the batch size and batch timeout in the ctor. @Hausmann, Steffen<mailto:shau...@amazon.de> knows more. 2. I guess your question is if current AsyncIO is not suff

Re: [DISCUSS] FLIP-171: Async Sink

2021-07-16 Thread Hausmann, Steffen
ssing this issue :) Best, Piotrek wt., 29 cze 2021 o 17:58 Hausmann, Steffen mailto:shau...@amazon.de>> napisał(a): Hey Poitr, I've just adapted the FLIP and changed the signature for the `submitRequestEntries` method: protected abstract void submitRequestEntries(List requestEntries, Resu

Re: [DISCUSS] FLIP-171: Async Sink

2021-06-29 Thread Hausmann, Steffen
On 25.06.21, 17:05, "Hausmann, Steffen" wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Hi Piotr, I’m happy to take your guidance on this. I nee

[RESULT][VOTE] FLIP-171: Async Sink

2021-06-25 Thread Hausmann, Steffen
Hi there, The voting time of FLIP-171 [1] has passed. I'm closing the vote now. There were four +1 votes, all of which are binding: - Arvid Heise (binding) - Danny Cranmer (binding) - 刘建刚 (binding) - Piotr Nowojski (binding) Poitr had some smaller comments on the implementation details of the

Re: [DISCUSS] FLIP-171: Async Sink

2021-06-25 Thread Hausmann, Steffen
nk wants to cancel the operation). I think this makes the contract a bit clearer. I think starting simple and then extending the API as we see the need is a good idea. Cheers, Till On Tue, Jun 22, 2021 at 11:20 AM Hausmann, Steffen mailto:shau...@amazon.de>> wrote: Hey, Agreed on starti

[VOTE] FLIP-171: Async Sink

2021-06-22 Thread Hausmann, Steffen
Hi there, After the discussion in [1], I’d like to open a voting thread for FLIP-171 [2], which proposes a base implementation for sinks that support async requests. The vote will be open until June 25 (72h), unless there is an objection or not enough votes. Cheers, Steffen [1]

Re: [DISCUSS] FLIP-171: Async Sink

2021-06-22 Thread Hausmann, Steffen
base/src/main/java/org/apache/flink/connector/base/source/reader/fetcher/SplitFetcher.java#L258-L258 > On Thu, Jun 10, 2021 at 10:09 AM Hausmann, Steffen > wrote: > >> Hey Piotrek, >> >> Thanks for your comments on the FLIP. I'll address your second qu

Re: [DISCUSS] FLIP-171: Async Sink

2021-06-16 Thread Hausmann, Steffen
, removing this would substantially reduce application size and clean the dependency chain Thanks, On 10/06/2021, 09:09, "Hausmann, Steffen" wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm

Re: [ANNOUNCE] New PMC member: Arvid Heise

2021-06-16 Thread Hausmann, Steffen
Congrats, Arvid! On 16.06.21, 13:22, "Dian Fu" wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Congratulations, Arvid! > 2021年6月16日 下午7:16,Benchao Li

Re: [DISCUSS] FLIP-171: Async Sink

2021-06-10 Thread Hausmann, Steffen
43-Unified-Sink-API-tp44602p44872.html [2] http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-143-Unified-Sink-API-tp44602p44930.html śr., 9 cze 2021 o 09:49 Hausmann, Steffen napisał(a): > Hi there, > > We would like to start a discuss

[DISCUSS] FLIP-171: Async Sink

2021-06-09 Thread Hausmann, Steffen
Hi there, We would like to start a discussion thread on "FLIP-171: Async Sink" [1], where we propose to create a common abstraction for destinations that support async requests. This abstraction will make it easier to add destinations to Flink by implementing a lightweight shim, while it