Re: Keyed CEP checkpoint fails

As @Kostas asked in your previous thread would be possible for you to share your code for that job or at least a minimal example to reproduce this behaviour. I fear we won’t be able to help you without any further info. Regards, Dawid > On 10 Aug 2017, at 14:10, Daiqing Li <lida

Re: CEP with Kafka source

the events upon arrival of it, we would not be able to produce proper results. I don’t know how does your text-file approach looks like, but if it does work differently I would assume you do not work in EventTime. Regards, Dawid > On 4 Aug 2017, at 09:40, Björn Hedström <bjorn.e.hedst...@gma

Re: CEP condition expression and its event consuming strategy

21:54, Chao Wang <chaow...@wustl.edu> wrote: > > Hi Dawid, > > Thank you. > > Ad. 1 I noticed that the method getEventsForPattern() returns an Iterable > and we need to further invoke .operator().next() to get access to the event > value. > > Ad. 2

Re: CEP condition expression and its event consuming strategy

ern("first")); } }). Ad. 2 Unfortunately right now as you said Pattern restarts each other event and it is not possible to change that strategy. There is ongoing work to introduce AfterMatchSkipStrategy[1], but at best it will be merged in 1.4.0. I did not give it much

Re: notNext() and next(negation) not yielding same output in Flink CEP

the A’s there should be no A. I hope this helps understanding how notNext works. Regards, Dawid > On 22 Jul 2017, at 20:32, Yassine MARZOUGUI <y.marzou...@mindlytix.com> wrote: > > Hi all, > > I would like to match the maximal consecutive sequences of events of type A

Re: Getting Errors when using keyby()

Hi Sridhar, Your class is missing default constructor(without arguments) thus it is not a valid POJO in Flink. You can check the requirements for POJO in link here: https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/api_concepts.html#pojos > On 12 Jul 2017, at 19:54, Sridhar

Re: Should customized Complex Events be Serializable?

What do you mean by ComplexEvents? Do you mean that the output of CEP library is DataStream? If so, then yes, they should be either Serializable or you should provide custom TypeSerializer. > On 12 Jul 2017, at 06:58, Sridhar Chellappa wrote: > > Folks, > > I am using

Re: Queries regarding FlinkCEP

eniami! / Cheers! Dawid Wysakowicz *Data/Software Engineer* Skype: dawid_wys | Twitter: @OneMoreCoder <http://getindata.com/> 2017-06-20 14:00 GMT+02:00 Biplob Biswas <revolutioni...@gmail.com>: > Hi dawid, > > Yes I am reading from multiple topics and yes a few topics

Re: Flink CEP not emitting timed out events properly

It is possible that two Watermarks will be passed through the Stream, but at an operator that has more than one input streams(in your case from different topics and partitions) the smallest Watermark is considered valid and passed downstream. Z pozdrowieniami! / Cheers! Dawid Wysakowicz *Data

Re: Queries regarding FlinkCEP

e is taken as a "global" Watermark. Z pozdrowieniami! / Cheers! Dawid Wysakowicz *Data/Software Engineer* Skype: dawid_wys | Twitter: @OneMoreCoder <http://getindata.com/> 2017-06-08 15:35 GMT+02:00 Biplob Biswas <revolutioni...@gmail.com>: > Hi, > > Can anyone che

Re: Flink CEP not emitting timed out events properly

Hi Biplop, If an operator has two inputs, the resulting Watermark is the smallest one from the two upstreams. More on that you can check here .

Re: FlinkCEP 1.3 scala, cannot apply select function

endEvent > } Z pozdrowieniami! / Cheers! Dawid Wysakowicz *Data/Software Engineer* Skype: dawid_wys | Twitter: @OneMoreCoder <http://getindata.com/> 2017-06-19 9:35 GMT+02:00 Sonex <alfredjens...@gmail.com>: > Hello I have created a simple pattern with FlinkCEP 1.3 as wel

Re: Stream sql example

ill file a JIRA for the previous issue tomorrow. Z pozdrowieniami! / Cheers! Dawid Wysakowicz *Data/Software Engineer* Skype: dawid_wys | Twitter: @OneMoreCoder <http://getindata.com/> 2017-06-09 22:25 GMT+02:00 Timo Walther <twal...@apache.org>: > Hi David, > > I think the proble

Re: Stream sql example

Sorry forgot to add the link: https://gist.github.com/dawidwys/537d12a6f2355cba728bf93f1af87b45 Z pozdrowieniami! / Cheers! Dawid Wysakowicz *Data/Software Engineer* Skype: dawid_wys | Twitter: @OneMoreCoder <http://getindata.com/> 2017-06-09 20:19 GMT+02:00 Dawid Wysakowicz <wysa

Stream sql example

e.flink.table.plan.nodes.datastream.StreamTableSourceScan.convertToInternalRow(StreamTableSourceScan.scala:35) > > at >> org.apache.flink.table.plan.nodes.datastream.StreamTableSourceScan.translateToPlan(StreamTableSourceScan.scala:107) > > You can check the source code here: Z pozdrowieniami!

Re: Problem with WebUI

nio.NioEventLoop.run(NioEventLoop.java:450) > at > io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) > at > io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144) > at java.lang.Thread.run(Thread

Problem with WebUI

is accessible and when I attach the same job to the running session everything is fine (the UI is still accessible) and the job runs correctly. Any ideas what may be the problem? Z pozdrowieniami! / Cheers! Dawid Wysakowicz *Data/Software Engineer* Skype: dawid_wys | Twitter: @OneMoreCoder

Re: Queries regarding FlinkCEP

amp *t + maxOutOfOrderness*. Try adding event like: 12,b,7,6 to your test set and some alerts should be generated. Z pozdrowieniami! / Cheers! Dawid Wysakowicz *Data/Software Engineer* Skype: dawid_wys | Twitter: @OneMoreCoder <http://getindata.com/> 2017-06-06 15:17 GMT+02:00 Bipl

Re: Queries regarding FlinkCEP

dev/event_time.html#event-time Z pozdrowieniami! / Cheers! Dawid Wysakowicz *Data/Software Engineer* Skype: dawid_wys | Twitter: @OneMoreCoder <http://getindata.com/> 2017-06-02 18:22 GMT+02:00 Till Rohrmann <trohrm...@apache.org>: > Hi Biplob, > > 1. The CEPPatter

Re: No Alerts with FinkCEP

Hi Biplob, The message you mention should not be a problem here. It just says you can't use your events as POJOs (e.g. you can't use keyBy("chargedAccount") ). Your code seems fine and without some example data I think it will be hard to help you. As for the PART 2 of your first email. In 1.3 we

Re: FlinkCEP latency/throughput

ch (ending pattern) and actual time when the match is emitted(for that case a select function is a good place I think). I think Kostas was also referring to similar kind of issue. Hope it will be helpful. Z pozdrowieniami! / Cheers! Dawid Wysakowicz *Data/Software Engineer* Skype: dawid_wys

Re: CEP memory requirements

Yes you are right, prior to 1.3.0 the state per key was never cleared. Right now due to FLINK-5174 <https://issues.apache.org/jira/browse/FLINK-5174>, in master branch, it is stored only if necessary. Z pozdrowieniami! / Cheers! Dawid Wysakowicz *Data/Software Engineer* Skype: daw

Re: CEP timeout occurs even for a successful match when using followedBy

event can be matched and you can still use SKIP TILL ANY by using followedByAny. I believe the SKIP TILL NEXT strategy is the one you expected. You can check it on master branch. We did introduce lots of new features and bugfixes to CEP for 1.3.0 version so any comments, tests or suggestions are

Re: CEP join across events

lue_b=X) both events of type = 1 create a seperate pattern branch and the event with type = 2 will be checked for a match twice for both of those branches. Regards, Dawid 2017-04-26 7:48 GMT+02:00 Elias Levy <fearsome.lucid...@gmail.com>: > There doesn't appear to be a way to join events acro

Re: Grafana plug-in for Flink

Hi, As far as I am aware Jamie used the example json datasource for grafana https://github.com/grafana/simple-json-datasource . At least I used it when I recreated his example for some introductory purposes. You can browse my example here: https://github.com/dawidwys/flink-intro. Best, Dawid

Starting flink HA cluster with start-cluster.sh

else HIGH_AVAILABILITY="none" fi fi if value "zookeeper" is read from config file the variable will be reset to "none" with the else branch. I just want to confirm it is a bug before filing a JIRA. Regards Dawid

Re: Issues with Event Time and Kafka

org/projects/flink/flink-docs-release-1.2/dev/event_time.html#event-time-and-watermarks> . In your case as you say that events can "lag" for 30 minutes, you should try the BoundedOutOfOrdernessTimestampExtractor. It is designed exactly for a case like yours. Regards, Dawid 2017-03-07

Re: Issues with Event Time and Kafka

Hi Max, How do you assign timestamps to your events (in event-time case)? Could you post whole code for your TimestampAndWatermarkAssigner? Regards, Dawid 2017-03-07 20:59 GMT+01:00 ext.mwalker <ext.mwal...@riotgames.com>: > Hi Stephan, > > The right number of events seem to l

Re: FlinkKafkaProducer usage

Have a look at https://github.com/apache/beam/blob/master/sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java . 02.02.2017 1:07 AM "Boris Lublinsky" napisał(a): > I am trying to write a quick sample of streaming word count using Beam > APIs

Re: Queryable State

Hi Nico, No problem at all, I've already presented my showcase with ValueStateDescriptor. Anyway, if I could help you somehow with the Queryablestate let me know. I will be happy to contribute some code. 2017-01-25 14:47 GMT+01:00 Nico Kruber <n...@data-artisans.com>: > Hi Dawid

Re: Queryable State

alizer.serializeList As I think they are not used at all even right now. Thanks for your time. Regards Dawid Wysakowicz 2017-01-16 13:25 GMT+01:00 Nico Kruber <n...@data-artisans.com>: > Hi Dawid, > regarding the original code, I couldn't reproduce this with the Java code I >

Re: Queryable State

may be right there might not be any production use for this state and it should be removed. Maybe the problem is just with the ListState and removing it would resolve also my problem :) Regards Dawid Wysakowicz 2017-01-13 18:50 GMT+01:00 Nico Kruber <n...@data-artisans.com>: > Hi Dawid, &

Re: Queryable State

Hey Ufuk. Did you maybe had a while to have a look at that problem? 2017-01-09 10:47 GMT+01:00 Ufuk Celebi <u...@apache.org>: > Hey Dawid! Thanks for reporting this. I will try to have a look over > the course of the day. From a first impression, this seems like a bug > to me. &g

Queryable State

lient.deserializeResponse(QueryClient.scala:44) You can browse the exact code at: https://github.com/dawidwys/flink-intro I would be grateful for any advice. Regards Dawid Wysakowicz

Re: Strange behaviour of windows

Thanks for the explanation. That was really stupid mistake from my side. By the way, I really like the whole idea and API. Really good job! Regards Dawid 2015-12-08 12:30 GMT+01:00 Aljoscha Krettek <aljos...@apache.org>: > Hi, > an important concept of the Flink API is that transf

Strange behaviour of windows

is empty). Do I understand the mechanism correctly and should my code work as I described? If not could you please explain a little bit? The code I've attached to this email. I would be grateful. Regards Dawid Wysakowicz import java.util.concurrent.TimeUnit import

Re: Strange behaviour of windows

Forgot to mention. I've checked it both on 0.10 and current master. 2015-12-07 20:32 GMT+01:00 Dawid Wysakowicz <wysakowicz.da...@gmail.com>: > Hi, > > I have recently experimented a bit with windowing and event-time mechanism > in flink and either I do not understand

<    1   2   3   4   5