Re: Questions about FoldFunction and WindowFunction

2016-11-02 Thread Aljoscha Krettek
Hi Yassine,
I made you a contributor in the Flink Jira so you will be able to assign
issues to yourself in the future. I also assigned this issue to you.

I think you only need to do changes in WindwedStream and AllWindowedStream.
Let me know if you need anything. :-)

Cheers,
Aljoscha

On Wed, 2 Nov 2016 at 14:48 Yassine MARZOUGUI 
wrote:

> Yes, with please. Could you please assign it temporarily to me? (I am not
> very familiar with the internal components of Flink and migh take some time
> before contributing the code, if by the time you are ready to work on it I
> am not yet done, you can reassign it to yourself)
>
> 2016-11-02 14:07 GMT+01:00 Aljoscha Krettek :
>
> Would you be interested in contributing a fix for that? Otherwise I'll
> probably fix work on that in the coming weeks.
>
> On Wed, 2 Nov 2016 at 13:38 Yassine MARZOUGUI 
> wrote:
>
> Thank you Aljoscha for your quick response.
>
> Best,
> Yassine
>
> 2016-11-02 12:30 GMT+01:00 Aljoscha Krettek :
>
> Hi Yassine,
>
> regarding 1. The close() method of the RichFoldFunction will only be
> called at the very end of your streaming job, so in practise it will never
> be called. This is there because of batch jobs, where you have an actual
> end in your processing.
>
> regarding 2. I'm afraid you came across a bug:
> https://issues.apache.org/jira/browse/FLINK-3869. We can't change this
> right now because we cannot break API instability but right at the end of
> this issue I'm proposing a different solution that we'll hopefully get in
> for the next release.
>
> Cheers,
> Aljoscha
>
> On Wed, 2 Nov 2016 at 10:42 Yassine MARZOUGUI 
> wrote:
>
> Hi all,
>
> I have a couple questions about FoldFunction and WindowFunction:
>
> 1. When using a RichFoldFunction after a window as in 
> keyedStream.window().fold(new
> RichFoldFunction()), is the close() method called after each window or
> after all the windows for that key are fired?
>
> 2. When applying a FoldFunction to a window followed by a WindowFunction
> via apply
> (R
>  initialValue,
> FoldFunction
> 
>  ,R>
>  foldFunction,
> WindowFunction
> 
>  
> ,W
> 
> > function), why should the output of the WindowFunction be of the same
> type as the input? It would be practical to have a different output type
> sometimes, for example one would fold tuples in the FoldFunction and then
> process the (only) aggregated tuple in the Window function and emit an
> Integer.
>
> Best,
> Yassine
>
>
>
>


Re: Questions about FoldFunction and WindowFunction

2016-11-02 Thread Yassine MARZOUGUI
Yes, with please. Could you please assign it temporarily to me? (I am not
very familiar with the internal components of Flink and migh take some time
before contributing the code, if by the time you are ready to work on it I
am not yet done, you can reassign it to yourself)

2016-11-02 14:07 GMT+01:00 Aljoscha Krettek :

> Would you be interested in contributing a fix for that? Otherwise I'll
> probably fix work on that in the coming weeks.
>
> On Wed, 2 Nov 2016 at 13:38 Yassine MARZOUGUI 
> wrote:
>
>> Thank you Aljoscha for your quick response.
>>
>> Best,
>> Yassine
>>
>> 2016-11-02 12:30 GMT+01:00 Aljoscha Krettek :
>>
>> Hi Yassine,
>>
>> regarding 1. The close() method of the RichFoldFunction will only be
>> called at the very end of your streaming job, so in practise it will never
>> be called. This is there because of batch jobs, where you have an actual
>> end in your processing.
>>
>> regarding 2. I'm afraid you came across a bug: https://issues.apache.
>> org/jira/browse/FLINK-3869. We can't change this right now because we
>> cannot break API instability but right at the end of this issue I'm
>> proposing a different solution that we'll hopefully get in for the next
>> release.
>>
>> Cheers,
>> Aljoscha
>>
>> On Wed, 2 Nov 2016 at 10:42 Yassine MARZOUGUI 
>> wrote:
>>
>> Hi all,
>>
>> I have a couple questions about FoldFunction and WindowFunction:
>>
>> 1. When using a RichFoldFunction after a window as in
>> keyedStream.window().fold(new RichFoldFunction()), is the close() method
>> called after each window or after all the windows for that key are fired?
>>
>> 2. When applying a FoldFunction to a window followed by a WindowFunction
>> via apply
>> (R
>>  initialValue,
>> FoldFunction
>> 
>> > 
>> ,R> foldFunction, WindowFunction
>> 
>> > 
>> ,W
>> 
>> > function), why should the output of the WindowFunction be of the same
>> type as the input? It would be practical to have a different output type
>> sometimes, for example one would fold tuples in the FoldFunction and then
>> process the (only) aggregated tuple in the Window function and emit an
>> Integer.
>>
>> Best,
>> Yassine
>>
>>
>>


Re: Questions about FoldFunction and WindowFunction

2016-11-02 Thread Aljoscha Krettek
Would you be interested in contributing a fix for that? Otherwise I'll
probably fix work on that in the coming weeks.

On Wed, 2 Nov 2016 at 13:38 Yassine MARZOUGUI 
wrote:

> Thank you Aljoscha for your quick response.
>
> Best,
> Yassine
>
> 2016-11-02 12:30 GMT+01:00 Aljoscha Krettek :
>
> Hi Yassine,
>
> regarding 1. The close() method of the RichFoldFunction will only be
> called at the very end of your streaming job, so in practise it will never
> be called. This is there because of batch jobs, where you have an actual
> end in your processing.
>
> regarding 2. I'm afraid you came across a bug:
> https://issues.apache.org/jira/browse/FLINK-3869. We can't change this
> right now because we cannot break API instability but right at the end of
> this issue I'm proposing a different solution that we'll hopefully get in
> for the next release.
>
> Cheers,
> Aljoscha
>
> On Wed, 2 Nov 2016 at 10:42 Yassine MARZOUGUI 
> wrote:
>
> Hi all,
>
> I have a couple questions about FoldFunction and WindowFunction:
>
> 1. When using a RichFoldFunction after a window as in 
> keyedStream.window().fold(new
> RichFoldFunction()), is the close() method called after each window or
> after all the windows for that key are fired?
>
> 2. When applying a FoldFunction to a window followed by a WindowFunction
> via apply
> (R
>  initialValue,
> FoldFunction
> 
>  ,R>
>  foldFunction,
> WindowFunction
> 
>  
> ,W
> 
> > function), why should the output of the WindowFunction be of the same
> type as the input? It would be practical to have a different output type
> sometimes, for example one would fold tuples in the FoldFunction and then
> process the (only) aggregated tuple in the Window function and emit an
> Integer.
>
> Best,
> Yassine
>
>
>


Re: Questions about FoldFunction and WindowFunction

2016-11-02 Thread Yassine MARZOUGUI
Thank you Aljoscha for your quick response.

Best,
Yassine

2016-11-02 12:30 GMT+01:00 Aljoscha Krettek :

> Hi Yassine,
>
> regarding 1. The close() method of the RichFoldFunction will only be
> called at the very end of your streaming job, so in practise it will never
> be called. This is there because of batch jobs, where you have an actual
> end in your processing.
>
> regarding 2. I'm afraid you came across a bug: https://issues.apache.
> org/jira/browse/FLINK-3869. We can't change this right now because we
> cannot break API instability but right at the end of this issue I'm
> proposing a different solution that we'll hopefully get in for the next
> release.
>
> Cheers,
> Aljoscha
>
> On Wed, 2 Nov 2016 at 10:42 Yassine MARZOUGUI 
> wrote:
>
>> Hi all,
>>
>> I have a couple questions about FoldFunction and WindowFunction:
>>
>> 1. When using a RichFoldFunction after a window as in
>> keyedStream.window().fold(new RichFoldFunction()), is the close() method
>> called after each window or after all the windows for that key are fired?
>>
>> 2. When applying a FoldFunction to a window followed by a WindowFunction
>> via apply
>> (R
>>  initialValue,
>> FoldFunction
>> 
>> > 
>> ,R> foldFunction, WindowFunction
>> 
>> > 
>> ,W
>> 
>> > function), why should the output of the WindowFunction be of the same
>> type as the input? It would be practical to have a different output type
>> sometimes, for example one would fold tuples in the FoldFunction and then
>> process the (only) aggregated tuple in the Window function and emit an
>> Integer.
>>
>> Best,
>> Yassine
>>
>


Re: Questions about FoldFunction and WindowFunction

2016-11-02 Thread Aljoscha Krettek
Hi Yassine,

regarding 1. The close() method of the RichFoldFunction will only be called
at the very end of your streaming job, so in practise it will never be
called. This is there because of batch jobs, where you have an actual end
in your processing.

regarding 2. I'm afraid you came across a bug:
https://issues.apache.org/jira/browse/FLINK-3869. We can't change this
right now because we cannot break API instability but right at the end of
this issue I'm proposing a different solution that we'll hopefully get in
for the next release.

Cheers,
Aljoscha

On Wed, 2 Nov 2016 at 10:42 Yassine MARZOUGUI 
wrote:

> Hi all,
>
> I have a couple questions about FoldFunction and WindowFunction:
>
> 1. When using a RichFoldFunction after a window as in 
> keyedStream.window().fold(new
> RichFoldFunction()), is the close() method called after each window or
> after all the windows for that key are fired?
>
> 2. When applying a FoldFunction to a window followed by a WindowFunction
> via apply
> (R
>  initialValue,
> FoldFunction
> 
>  ,R>
>  foldFunction,
> WindowFunction
> 
>  
> ,W
> 
> > function), why should the output of the WindowFunction be of the same
> type as the input? It would be practical to have a different output type
> sometimes, for example one would fold tuples in the FoldFunction and then
> process the (only) aggregated tuple in the Window function and emit an
> Integer.
>
> Best,
> Yassine
>