The new API forces iteration through every element of the buffer even if a
single value to be evicted. What about implementing Iterator.remove()
method for elements? The API would look like:
public interface Evictor<T, W extends Window> extends Serializable {
/**
* Optionally evicts elements. Called before windowing function.
*
* @param elements The elements currently in the pane. Use
Iterator.remove to evict.
* @param size The current number of elements in the pane.
* @param window The {@link Window}
*/
void evictBefore(Iterable<T> elements, int size, EvictorContext ctx);
/**
* Optionally evicts elements. Called after windowing function.
*
* @param elements The elements currently in the pane. Use
Iterator.remove to evict.
* @param size The current number of elements in the pane.
* @param window The {@link Window}
*/
void evictAfter(Iterable<T> elements, int size, EvictorContext ctx);
}
Such API allows to abort iteration at any point and evict elements in any
order.
Thanks,
Maxim.
On Wed, Jul 6, 2016 at 9:04 AM, Vishnu Viswanath <
vishnu.viswanat...@gmail.com> wrote:
>
> Hi Aljoscha,
>
> Thanks. Yes the new interface seems to address points 1 and 2. of
>
> *1) I am having a use case where I have to create a custom Evictor that
> will evict elements from the window based on the value (e.g., if I have
> elements are of case class Item(id: Int, type:String) then evict elements
> that has type="a"). I believe this is not currently possible.*
> *2) this is somewhat related to 1) where there should be an option to
evict
> elements from anywhere in the window. not only from the beginning of the
> window. (e.g., apply the delta function to all elements and remove all
> those don't pass. I checked the code and evict method just returns the
> number of elements to be removed and processTriggerResult just skips those
> many elements from the beginning. *
> *3) Add an option to enables the user to decide if the eviction should
> happen before the apply function or after the apply function. Currently it
> is before the apply function, but I have a use case where I need to first
> apply the function and evict afterward.*
>
> I would be interested in contributing to the code base. Please let me know
> the steps.
>
> Thanks and Regards,
> Vishnu Viswanath
>
> On Wed, Jul 6, 2016 at 11:49 AM, Aljoscha Krettek <aljos...@apache.org>
> wrote:
>
> > Hi,
> > as mentioned in the thread on improving the Windowing API I also have a
> > design doc just for improving WindowEvictors. I had this in my head for
a
> > while but was hesitant to publish but since people are asking about this
> > now might be a good time to post it. Here's the doc:
> >
https://docs.google.com/document/d/1Rr7xzlItYqvFXLyyy-Yv0vvw8f29QYAjm5i9E4A_JlU/edit?usp=sharing
> >
> > Feedback/Suggestions are very welcome! Please let me know what you
think.
> >
> > @Vishnu: Are you interested in contributing a solution for this to the
> > Flink code base? I'd be very happy to work with you on this.
> >
> > Cheers,
> > Aljoscha
> >
> > P.S. I think it would be best to keep discussions to the ML because
> > comments on the doc will not be visible here for everyone.
> >
