Hi Kostas,
Thank you for your reply, yes that will be a good functionality to have,
but for now the Custom Trigger as close to 1.0.3 works for me.
public TriggerResult onElement(Object element, long timestamp, TimeWindow
window, TriggerContext ctx) throws Exception { if(window.maxTimestamp() <=
ctx.getCurrentWatermark()) { *return TriggerResult.FIRE_AND_PURGE;* } else
{ ctx.registerEventTimeTimer(window.maxTimestamp()); return
TriggerResult.CONTINUE; } } public TriggerResult onEventTime(long time,
TimeWindow window, TriggerContext ctx) { return
*TriggerResult.FIRE_AND_PURGE;* }
This is the change I have done in my custom trigger which is similar to
1.0.3 , does this looks good to you ?
(I am not using canMerge and onMerge methods since I am not aware of it)
Regards,
Vinay Patil
On Tue, Oct 4, 2016 at 3:58 AM, Kostas Kloudas [via Apache Flink User
Mailing List archive.] <ml-node+s2336050n9307...@n4.nabble.com> wrote:
> Hello LF and Vinay,
>
> With the introduction of “allowed lateness” elements and windows are kept
> around until the watermark
> passes the window.maxTimestamp + allowed_lateness and then they are
> cleaned up (garbage collected)
>
> Every element that comes in and belongs to a window that is garbage
> collected is dropped as super-late.
> Elements that are late, but no more than the allowed lateness, they are
> kept the window fires as before.
>
> If you know what the maximum latency is, then the best way is to set the
> allowed lateness to that value.
>
> Currently Flink drops super-late elements and does not provide any
> mechanism to manually handle these
> elements, BUT there are discussions about adding such a functionality that
> will allow you to (probably) have
> a separate stream with only these elements.
>
> Thanks,
> Kostas
>
> On Oct 3, 2016, at 5:28 PM, [hidden email]
> <http:///user/SendEmail.jtp?type=node&node=9307&i=0> wrote:
>
> Not yet.
> I'm hoping a Flink export on this mailing list will reply.
>
>
> - LF
>
>
>
> ------------------------------
> *From:* vinay patil <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=9307&i=1>>
> *To:* [hidden email] <http:///user/SendEmail.jtp?type=node&node=9307&i=2>
> *Sent:* Monday, October 3, 2016 8:09 AM
> *Subject:* Re: Regarding Late Elements
>
> Hi LF,
>
> So did you manage to get the workaround for it ?
>
> I am using a Custom Trigger which is similar to 1.0.3 with few changes
>
> Regards,
> Vinay Patil
>
> On Mon, Oct 3, 2016 at 10:02 AM, lgfmt [via Apache Flink User Mailing List
> archive.] <[hidden email]> wrote:
>
> We have the same requirement - we cannot discard any data even if it
> arrives late.
>
>
> - LF
>
>
>
>
> ------------------------------
> *From:* Vinay Patil <[hidden email]>
> *To:* [hidden email]
>
> *Sent:* Sunday, October 2, 2016 8:21 PM
> *Subject:* Regarding Late Elements
>
> Hi Guys,
>
> Just wanted to get an idea on Why Flink decided to completely discard late
> elements in the latest version ?, this was not the case in 1.0.3
>
>
> P.S In our case the data is critical so we cannot discard a single record
> even if it is late, I have written a custom trigger (as suggested by
> Aljoscha) to even accept late elements.
>
>
> Regards,
> Vinay Patil
>
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-flink-user- mailing-list-archive.2336050.
> n4.nabble.com/Regarding-Late- Elements-tp9284p9292.html
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Regarding-Late-Elements-tp9284p9292.html>
> To start a new topic under Apache Flink User Mailing List archive., email
> [hidden
> email]
> To unsubscribe from Apache Flink User Mailing List archive., click here.
> NAML
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>
>
> ------------------------------
> View this message in context: Re: Regarding Late Elements
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Regarding-Late-Elements-tp9284p9294.html>
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/> at
> Nabble.com <http://nabble.com>.
>
>
>
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/Regarding-Late-Elements-tp9284p9307.html
> To start a new topic under Apache Flink User Mailing List archive., email
> ml-node+s2336050n1...@n4.nabble.com
> To unsubscribe from Apache Flink User Mailing List archive., click here
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=dmluYXkxOC5wYXRpbEBnbWFpbC5jb218MXwxODExMDE2NjAx>
> .
> NAML
> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Regarding-Late-Elements-tp9284p9323.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
