Re: Windows on SinkFunctions
Hey, In your original email, you wrote: Because if I have multiple sinks that that only for one of them I need a > Window, the second solution might be problematic. You can also send the data of an operator to multiple sinks Source --> MyComputationProcessFunction --> DataBatcher --> BatchedSink \ \--> StreamingSink (also here if it renders weirdly in email: https://gist.github.com/rmetzger/9ef311c2926a82fdba2b2b7af9ad65d6 ) In this example "MyComputationProcessFunction" is sending the data to two downstream operators: DataBatcher and StreamingSink. For the "DataBatcher", you can also consider building something yourself with ProcessFunction. I would just collect the data in a List, and emit it once the list reached 500 elements, or a custom 5 minute trigger has triggered. On Sun, Mar 29, 2020 at 12:07 PM Sidney Feiner wrote: > Thanks! > What am I supposed to put in the apply/process function for the sink to be > invoked on a List of items? > > *Sidney Feiner* */* Data Platform Developer > M: +972.528197720 */* Skype: sidney.feiner.startapp > > [image: emailsignature] > > -- > *From:* tison > *Sent:* Sunday, March 22, 2020 4:19 PM > *To:* Sidney Feiner > *Cc:* user@flink.apache.org > *Subject:* Re: Windows on SinkFunctions > > Hi Sidney, > > For the case, you can exactly write > > stream. > ... > .window() > .apply() > .addSink() > > Operator chain will chain these operators into one so that you don't have > to worry about the efficiency. > > Best, > tison. > > > Sidney Feiner 于2020年3月22日周日 下午10:03写道： > > Hey, > I wanted to know if it's possible to define a SinkFunction as a > WindowFunction as well. > For example, I would like the sink to be invoked every 5 minute or once > 500 events reached the sink. > Is there a way to do this inside the sink implementation? Or do I have to > create the windows prior in the pipeline? > Because if I have multiple sinks that that only for one of them I need a > Window, the second solution might be problematic. > > Thanks :) > > *Sidney Feiner* */* Data Platform Developer > M: +972.528197720 */* Skype: sidney.feiner.startapp > > [image: emailsignature] > >
Re: Windows on SinkFunctions
Thanks! What am I supposed to put in the apply/process function for the sink to be invoked on a List of items? Sidney Feiner / Data Platform Developer M: +972.528197720 / Skype: sidney.feiner.startapp [emailsignature] From: tison Sent: Sunday, March 22, 2020 4:19 PM To: Sidney Feiner Cc: user@flink.apache.org Subject: Re: Windows on SinkFunctions Hi Sidney, For the case, you can exactly write stream. ... .window() .apply() .addSink() Operator chain will chain these operators into one so that you don't have to worry about the efficiency. Best, tison. Sidney Feiner mailto:sidney.fei...@startapp.com>> 于2020年3月22日周日 下午10:03写道： Hey, I wanted to know if it's possible to define a SinkFunction as a WindowFunction as well. For example, I would like the sink to be invoked every 5 minute or once 500 events reached the sink. Is there a way to do this inside the sink implementation? Or do I have to create the windows prior in the pipeline? Because if I have multiple sinks that that only for one of them I need a Window, the second solution might be problematic. Thanks :) Sidney Feiner / Data Platform Developer M: +972.528197720 / Skype: sidney.feiner.startapp [emailsignature]
Re: Windows on SinkFunctions
Hi Sidney, For the case, you can exactly write stream. ... .window() .apply() .addSink() Operator chain will chain these operators into one so that you don't have to worry about the efficiency. Best, tison. Sidney Feiner 于2020年3月22日周日 下午10:03写道： > Hey, > I wanted to know if it's possible to define a SinkFunction as a > WindowFunction as well. > For example, I would like the sink to be invoked every 5 minute or once > 500 events reached the sink. > Is there a way to do this inside the sink implementation? Or do I have to > create the windows prior in the pipeline? > Because if I have multiple sinks that that only for one of them I need a > Window, the second solution might be problematic. > > Thanks :) > > *Sidney Feiner* */* Data Platform Developer > M: +972.528197720 */* Skype: sidney.feiner.startapp > > [image: emailsignature] > >
Windows on SinkFunctions
Hey, I wanted to know if it's possible to define a SinkFunction as a WindowFunction as well. For example, I would like the sink to be invoked every 5 minute or once 500 events reached the sink. Is there a way to do this inside the sink implementation? Or do I have to create the windows prior in the pipeline? Because if I have multiple sinks that that only for one of them I need a Window, the second solution might be problematic. Thanks :) Sidney Feiner / Data Platform Developer M: +972.528197720 / Skype: sidney.feiner.startapp [emailsignature]
