Re: CEP issue

rstAfterLast ). > > We will keep instrumenting the code ( which apart from the null message is > easily understandable ) but would love to hear your feedback. > > > > > > > > > > > > > > > > > > > > > > &

Re: Simple CEP pattern

Hi Esa, You could try the examples either from the documentation or from the training. http://training.data-artisans.com/exercises/CEP.html Kostas > On Mar 7, 2018, at 11:32 AM, Esa Heikkinen > wrote: > >

Re: Questions about the FlinkCEP

the value to (global) variable for later use in the same pattern or > even other places in the application. Is this possible ? > > Best, Esa > > From: Kostas Kloudas [mailto:k.klou...@data-artisans.com] > Sent: Thursday, March 1, 2018 11:35 AM > To: Esa Heikkinen <esa

Re: Questions about the FlinkCEP

Hi Esa, The answers to the questions are inlined. > On Feb 28, 2018, at 8:32 PM, Esa Heikkinen wrote: > > Hi > > I have tried to learn FlinkCEP [1], but i have yet not found the clear > answers for questions: > 1) Whether the pattern of CEP is meant only for one data

Re: Important (proposed) CEP changes for Flink 1.5.

4. I do hope that all other pipes will > remain back ward compatible ( 1.4 to 1.5 ). > > On Wed, Feb 21, 2018 at 5:06 AM, Kostas Kloudas <kklou...@gmail.com > <mailto:kklou...@gmail.com>> wrote: > Hi all, > > Currently due to backwards compatibility there

Re: Optimizing multiple aggregate queries on a CEP using Flink

Hi Sahil, Currently CEP does not support multi-query optimizations out-of-the-box. In some cases you can do manual optimizations to your code, but there is no optimizer involved. Cheers, Kostas > On Feb 15, 2018, at 11:12 AM, Sahil Arora wrote: > > Hi Timo, > Thanks

Re: CEP for time series in csv-file

Hi Esa, I think the best place to start is the documentation available at the flink website. Some pointers are the following: CEP documentation: https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/libs/cep.html

Re: Flink CEP exception during RocksDB update

Hi Varun, The branch I previously sent you has been now merged to the master. So could you try the master and tell us if you see any change in the behavior? Has the problem been fixed, or has the message of the exception changed? Thanks, Kostas > On Jan 29, 2018, at 10:09 AM, Kostas Klou

Re: CEP issue

Thanks a lot Vishal! We are looking forward to a test case that reproduces the failure. Kostas > On Feb 2, 2018, at 4:05 PM, Vishal Santoshi wrote: > > This is the pattern. Will create a test case. > /** > * > * @param condition a single condition is applied as

Re: Flink CEP exception during RocksDB update

let me know if the problem is still there? In addition, are you using Scala with case classes or Java? Thanks for helping fix the problem, Kostas > On Jan 24, 2018, at 5:54 PM, Kostas Kloudas <k.klou...@data-artisans.com> > wrote: > > Hi Varun, > > Thanks for taking time t

Re: Flink CEP exception during RocksDB update

y suggestions on further debugging runtime checkpoints would be > very helpful. > Thanks in advance for your assistance. > > Regards, > Varun > > On Jan 18, 2018, at 8:11 AM, Kostas Kloudas <k.klou...@data-artisans.com > <mailto:k.klou...@data-artisans.com>>

Re: CEP issue in 1.3.2. Does 1.4 fix this ?

Hi Vishal, Thanks for checking and glad to hear that your job works after the fix! As for the equals/hashcode question, if your question is if you have to implement exact equals() method and the corresponding hashcode() then the answer is yes. These methods are used when retrieving and cleaning

Re: Unable to query MapState

Hi Velu, I would recommend to switch to Flink 1.4 as the queryable state has been refactored to be compatible with all types of state. You can read more here: https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/queryable_state.html

Re: Flink CEP exception during RocksDB update

anks, > Varun > > Sent from my iPhone > > On Jan 15, 2018, at 10:21 AM, Kostas Kloudas <k.klou...@data-artisans.com > <mailto:k.klou...@data-artisans.com>> wrote: > >> Hi Varun, >> >> This can be related to this issue: >> https://issues.a

Re: Flink CEP exception during RocksDB update

Hi Varun, This can be related to this issue: https://issues.apache.org/jira/browse/FLINK-8226 which is currently fixed on the master. Could you please try the current master to see if the error persists? Thanks, Kostas > On Jan 15, 2018, at

Re: How to apply patterns from a source onto another datastream?

Hi Jayant, As Dawid said, currently dynamically updating patterns is currently not supported. There is also this question raised in the dev mailing list with the subject CEP: Dynamic Patterns. I will repeat my answer here so that we are on the same page: "To support this, we need 2 features

Re: Triggers in Flink CEP

Hi Shailesh, The pattern operator does not use Flink’s windowing mechanism internally. Conceptually you may think that there are windows in both, and this is true, but there are significant differences that prevent using Flink windowing for CEP. The above implies also that using triggers for

Re: Hardware Reference Architecture

liably-and-efficiently-operate-apache-flink/3 <http://www.slideshare.net/FlinkForward/flink-forward-berlin-2017-robert-metzger-keep-it-going-how-to-reliably-and-efficiently-operate-apache-flink/3> Kostas > On Dec 7, 2017, at 6:36 PM, Kostas Kloudas <k.klou...@data-artisans.com> &g

Re: Hardware Reference Architecture

Hi Hayden, It would be nice if you could share a bit more details about your use case and the load that you expect to have, as this could allow us to have a better view of your needs. As a general set of rules: 1) I would say that the bigger your cluster (in terms of resources, not necessarily

Re: Testing CoFlatMap correctness

Hi Tovi, What you need is the TwoInputStreamOperatorTestHarness. This will allow you to do something like: TwoInputStreamOperatorTestHarness testHarness = new TwoInputStreamOperatorTestHarness<>(myoperator); testHarness.setup(); testHarness.open();

Re: Maintain heavy hitters in Flink application

Hi Max, You are right that Queryable State is not designed to be used as a means for a job to query its own state. In fact, given that you do not know the jobId of your job from within the job itself, I do not think you can use queryable state in your scenario. What you can do is to have a

Re: Are there plans to support Hadoop 2.9.0 on near future?

/cwiki.apache.org/confluence/display/FLINK/Flink+Release+and+Feature+Plan#FlinkReleaseandFeaturePlan-Flink1.4> > that they estimated releasing 1.4 at September. Do you know if it will be > released this year or we may have to wait longer? > > Thanks a lot. > > De: Kostas

Re: Are there plans to support Hadoop 2.9.0 on near future?

Hi Oriol, As you may have seen form the mailing list we are currently in the process of releasing Flink 1.4. This is going to be a hadoop-free distribution which means that it should work with any hadoop version, including Hadoop 2.9.0. Given this, I would recommend to try out the release

Re: Bad entry in block exception with RocksDB

Hi Kien, Could you share some more information about your job? What operators are you using, the format of your elements, etc? Thanks, Kostas > On Nov 23, 2017, at 2:23 AM, Kien Truong wrote: > > Hi, > > We are seeing this exception in one of our job, whenever a

Re: readFile, DataStream

Hi Juan, The problem is that once a file for a certain timestamp is processed and the global modification timestamp is modified, then all files for that timestamp are considered processed. The solution is not to remove the = from the modificationTime <= globalModificationTime; in

Re: Queryable State Python

Hi Martin, I will try to reply to your questions inline: > On Nov 10, 2017, at 1:59 PM, Martin Eden wrote: > > Hi, > > Our team is looking at replacing Redis with Flink's own queryable state > mechanism. However our clients are using python. > > 1. Is there a python

Re: When using Flink for CEP, can the data in Cassandra database be used for state

Hi Shyla, Happy to hear that you are experimenting with CEP! For enriching your input stream with data from Cassandra (or whichever external storage system) you could use: * either the AsyncIO functionality offered by Flink

Re: FlinkCEP, circular references and checkpointing failures

Perfect! thanks a lot! Kostas > On Nov 3, 2017, at 3:23 PM, Federico D'Ambrosio > <federico.dambro...@smartlab.ws> wrote: > > Hi Kostas, > > yes, I'm using 1.3.2. I'll try the current master and I'll get back to you. > > 2017-11-03 15:21 GMT+01:00 Kostas Klouda

Re: FlinkCEP, circular references and checkpointing failures

Hi Federico, I assume that you are using Flink 1.3, right? In this case, in 1.4 we have fixed a bug that seems similar to your case: https://issues.apache.org/jira/browse/FLINK-7756 Could you try the current master to see if it fixes your

Re: FlinkCEP: pattern application on a KeyedStream

Hi Federico, If I understand your question correctly, then yes, the application of a Pattern on a keyed stream is similar to the application of a map function. It will search for the pattern on each per-key stream of data. So there will be state (buffer with partial matches, queued elements,

Re: async io operator timeouts

> ~ > Karthik > > On Mon, Oct 9, 2017 at 7:12 PM, Kostas Kloudas <k.klou...@data-artisans.com > <mailto:k.klou...@data-artisans.com>> wrote: > Hi Karthik, > > Currently there is no way to provide a handler for timed-out requests. > So the behavior i

Re: serialization error when using multiple metrics counters

Hi Colin, Are you initializing your counters from within the open() method of you rich function? In other words, are you calling counter = getRuntimeContext.getMetricGroup.counter(“my counter”) from within the open(). The counter interface is not serializable. So if you instantiate the

Re: Bucketing/Rolling Sink: How to overwrite method "openNewPartFile" - to append a new timestamp to part file path every time a new part file is being created

Hi Raja, To know about the method, I suppose you have looked at the source code of the sink. I think that including the timestamp of the element in the path file is not as easy as overriding the openNewPartFile. The reason is that the filenames serve as identities for the associated state of

Re: Windowed Stream Queryable State Support

Hi Vijay, If by “Windowed Stream Queryable State Support” you mean when will Flink allow to query the state of an in-flight window, then a version will be available in 1.4 yes. Cheers, Kostas > On Oct 7, 2017, at 2:55 PM, vijayakumar palaniappan > wrote: > > What

Re: Issue with CEP library

58873/flink-cep-not-recognizing-pattern> > > > > I would really appreciate your guidance on this. > > Best regards, > Ajay > > > > > > On Thu, Sep 28, 2017 at 1:38 AM, Kostas Kloudas <k.klou...@data-artisans.com > <mailto:k.klou

Re: Issue with CEP library

Hi Ajay, I will look a bit more on the issue. But in the meantime, could you run your job with parallelism of 1, to see if the results are the expected? Also could you change the pattern, for example check only for the start, to see if all keys pass through. As for the code, you apply

Re: CustomPartitioner that simulates ForwardPartitioner and watermarks

le and data serialization/deserialization. Since the stream is large I > want to avoid the network shuffle at the least. > > I thought operator instances within a taskmanager would get the same indexId, > but apparently this is not the case. > > Thanks, > >> O

Re: StreamCorruptedException

Hi Sridhar, From looking at your code: 1) The “KafkaDataSource” is a custom source that you implemented? Does this source buffer anything? 2) The getStreamSource2 seems to return again a "new KafkaDataSource”. Can this be a problem? 3) You are working on processing time and you are simply

Re: the design of spilling to disk

Hi Florin, Unfortunately, there is no design document. The UnilateralSortMerger.java is used in the batch processing mode (not is streaming) and, in fact, the code dates some years back. I cc also Fabian as he may have more things to say on this. Now for the streaming side, Flink uses 3

Re: Queryable State

Hi Navneeth, If you increase the timeout, everything works ok? I suppose from your config that you are running in standalone mode, right? Any other information about the job (e.g. code and/or size of state being fetched) and the cluster setup that can help us pin down the problem, would be

Re: QueryableState - No KvStateLocation found for KvState instance

Hi, As Biplob said this means that the JM cannot find the requested state. The reasons can be one of the above but given that you said you are using the FlinkMiniCluster, I assume you are testing. In this case, it can also be that you start querying your state to soon after the job is submitted,

Re: BucketingSink never closed

Hi Flavio, If I understand correctly, I think you bumped into this issue: https://issues.apache.org/jira/browse/FLINK-2646 There is also a similar discussion on the BucketingSink here:

Re: Access to datastream from BucketSink

hat-I-need-create-from-the-stream"); > } > } > > my question now is how do I access the data stream from within the S3Bucketer > so that I can generate a filename based on the data with the data stream. > > Thanks, > >> On 16 Aug 2017, at 12:55, Kostas

Re: Access to datastream from BucketSink

org/projects/flink/flink-docs-release-1.3/api/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.html > > <https://ci.apache.org/projects/flink/flink-docs-release-1.3/api/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.html> > > >>

Re: Access to datastream from BucketSink

Hi Ant, I think you can do it by implementing your own Bucketer. Cheers, Kostas . > On Aug 16, 2017, at 1:09 PM, ant burton wrote: > > Hello, > > Given > >// Set StreamExecutionEnvironment >final StreamExecutionEnvironment env = >

Re: FsStateBackend with incremental backup enable does not work with Keyed CEP

Hi Daiqing, I think Stefan is right and this will be fixed in the upcoming release. Could you open a JIRA for it with the Exception that you posted here? Thanks, Kostas > On Aug 12, 2017, at 10:05 AM, Stefan Richter > wrote: > > Hi, > > from a quick look, I

Re: Flink CEP issues

Hi Daiqing, Is it possible to share your job in order to reproduce the problem? Or at least a minimal example. If you see from the JIRA, there is another user in https://issues.apache.org/jira/browse/FLINK-6321 who had a similar problem but we

Re: [POLL] Dropping savepoint format compatibility for 1.1.x in the Flink 1.4.0 release

+1 > On Aug 2, 2017, at 3:16 PM, Till Rohrmann wrote: > > +1 > > On Wed, Aug 2, 2017 at 9:12 AM, Stefan Richter > wrote: > >> +1 >> >> Am 28.07.2017 um 16:03 schrieb Stephan Ewen : >> >> Seems like no one raised a concern

Re: data loss after implementing checkpoint

Hi Sridhar, Stephan already covered the correct sequence of actions in order for your second program to know its correct starting point. As far as the active/inactive rules are concerned, as Nico pointed out you have to somehow store in the backend which rules are active and which are not

Re: Integrating Flink CEP with a Rules Engine

The rules, or patterns supported by FlinkCEP are presented in the documentation link I posted earlier. Dynamically updating these patterns, is not supported yet, but there are discussions to add this feature soon. If the rules you need are supported by the current version of FlinkCEP, then

Re: Integrating Flink CEP with a Rules Engine

of the feasibility of this or can see some other > issues/consequences please comment. I also have the impression that > distribution is less of an issue because the rete network is > calculated only once and updates are not 'dynamic' (but I might be > wrong). > > Ismaël > >

Re: Integrating Flink CEP with a Rules Engine

Hi Jorn and Sridhar, It would be worth describing a bit more what these tools are and what are your needs. In addition, and to see what the CEP library already offers here you can find the documentation: https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/libs/cep.html

Re: Flink CEP not emitting timed out events properly

Are you sure that after incrementing the wm by 1sec, there is no element that will come with a timestamp smaller than this? Or, that after 10sec of inactivity, no element will come with such a timestamp? Kostas > On Jun 20, 2017, at 4:18 PM, Biplob Biswas wrote: > >

Re: Flink CEP not emitting timed out events properly

You are correct that elements are waiting until a watermark with a higher timestamp than theirs (or the patterns timeout) arrives. Now for the Watermark emitter, 1) how do you measure the 10sec in processing time and ii) by how much do you advance the watermark. If you advance it by a lot,

Re: Flink CEP not emitting timed out events properly

Hi Biplob, You are correct that only a higher watermark leads to discarded events. Are you sure that your custom watermark emitter does not emit a high watermark? E.g. your partition has elements that are far out-of-order. In addition, are you sure that your elements are not simply buffered and

Re: Flink CEP not emitting timed out events properly

Hi Biplob, If you know what you want, you can always write your custom AssignerWithPeriodicWatermarks that does your job. If you want to just increase the watermark, you could simply check if you have received any elements and if not, emit a watermark with the timestamp of the previous

Re: Flink CEP not emitting timed out events properly

Hi Biplob, With processing time there are no watermarks in the stream. The problem that you are seeing is because in processing time, the CEP library expects the “next” element to come, in order to investigate if some of the patterns have timed-out. Kostas > On Jun 16, 2017, at 1:29 PM,

Re: Java 8 lambdas for CEP patterns won't compile

Done. > On Jun 12, 2017, at 12:24 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > Can you add link to this thread in the JIRA ? > > Cheers > > On Mon, Jun 12, 2017 at 3:15 AM, Kostas Kloudas <k.klou...@data-artisans.com > <mailto:k.klou...@data-a

Re: Fink: KafkaProducer Data Loss

Hi Ninad, I think that Gordon could shed some more light on this but I suggest you should update your Flink version to at least the 1.2. The reason is that we are already in the process of releasing Flink 1.3 (which will come probably today) and a lot of things have changed/fixed/improved

Re: No Alerts with FinkCEP

You could also remove the autoWatermarkInterval, if you are satisfied with ProcessingTime. Although keep in mind that processingTime assigns timestamps to elements based on the order that they arrive to the operator. This means that replaying the same stream can give different results. If

Re: No Alerts with FinkCEP

Hi Biplob, Great to hear that everything worked out and that you are not blocked! For the timestamp assigning issue, you mean that you specified no timestamp extractor in your job and all your elements had Long.MIN_VALUE timestamp right? Kostas > On May 31, 2017, at 1:28 PM, Biplob Biswas

Re: Does RichFilterFunction work on multiple thread?

Your objects will be processed by a single thread. Kostas > On May 26, 2017, at 4:50 PM, luutuan wrote: > > Hi, when I have a set of objects goes through a RichFilterFunction, by > default, will the filter handle all objects in 1 single thread or will > divide the work to

Re: No Alerts with FinkCEP

Hi Biplob, For the 1.4 version, the input of the select function has changed to expect a list of matching events (Map map instead of Map map), as we have added quantifiers. Also the FIlterFunction has changed to SimpleCondition. The documentation is lagging a bit

Fwd: invalid type code: 00

I am forwarding Stefan’s reply: Hi, this problem can be caused by https://issues.apache.org/jira/browse/FLINK-6044. It is fixed in 1.2.1 and 1.3. Best, Stefan >> Am 26.05.2017 um 16:16 schrieb Kostas Kloudas <k.klou...@data-artisans.com>: >> >> Hi, >> >>

Fwd: invalid type code: 00

I am forwarding Stefan’s reply here: > Hi, > > this problem can be caused by > https://issues.apache.org/jira/browse/FLINK-6044 > <https://issues.apache.org/jira/browse/FLINK-6044>. It is fixed in 1.2.1 and > 1.3. > > Best, > Stefan > >> Am 26.05.20

Re: invalid type code: 00

Hi, Could you provide some info on when is this error happening? From what I see you are using the heap or fs state backend and you are failing to read the state back when restoring from a failure. The failure can be unrelated to this, but it could be useful if you could check the task manager

Re: No Alerts with FinkCEP

One additional comment, from your code it seems you are using Flink 1.2. It would be worth upgrading to 1.3. The updated CEP library includes a lot of new features and bugfixes. Cheers, Kostas > On May 26, 2017, at 3:33 PM, Kostas Kloudas <k.klou...@data-artisans.com> > wrote: &g

Re: No Alerts with FinkCEP

Hi Biplob, From a first scan of the code I cannot find sth fishy. You are working on ProcessingTime, given that you do not provide any time characteristic specification, right? In this case, if you print your partitionedInput stream, do you see elements flowing as expected? If elements are

Re: Are timers in ProcessFunction fault tolerant?

tolerant and maintain association with the key of the stream? > > On Fri, May 26, 2017 at 1:51 PM, Kostas Kloudas <k.klou...@data-artisans.com > <mailto:k.klou...@data-artisans.com>> wrote: > Hi Moiz, > > state.clear() refers to the state that you have reg

Re: invalid type code: 00

Hi! Can you give us some information about your job? Which Flink version you are using? What is your job doing (e.g. operators that you are using)? Which operator throws this exception? Which state-backend are you using? This exception means that you cannot retrieve your state because of

Re: Are timers in ProcessFunction fault tolerant?

Hi Moiz, state.clear() refers to the state that you have registered in your job, using the getState() from the runtimeContext. Timers are managed by Flink’s timer service and they are cleaned up by Flink itself when the job terminates. Kostas > On May 26, 2017, at 6:41 AM, Moiz S Jinia

Re: Question about start with checkpoint.

Hi, In order to change parallelism, you should take a savepoint, as described here: https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/savepoints.html Kostas > On May 21, 2017, at 5:43 AM,

Re: FlinkCEP latency/throughput

Hello Alfred, As a first general remark, Flink was not optimized for multicore deployments but rather for distributed environments. This implies overheads (serialization, communication etc), when compared to libs optimized for multicores. So there may be libraries that are better optimized for

Re: Timer fault tolerance in Flink

Hi Rahul, The timers are fault tolerant and their timestamp is the absolute value of when to fire. This means that if you are at time t = 10 and you register a timer “10 ms from now”, the timer will have a firing timestamp of 20. This is checkpointed, so the “new machine” that takes over the

Re: Stateful streaming question

ou can do this > with one job, two jobs are not needed. At every incoming event, check the > previous state and update/output to kafka or whatever data store you are > using. > > > > > > Thanks > > Ankit > > > > From: Flavio Pompermai

Re: Stateful streaming question

Hi Flavio, From what I understand, for the first part you are correct. You can use Flink’s internal state to keep your enriched data. In fact, if you are also querying an external system to enrich your data, it is worth looking at the AsyncIO feature:

Re: Problem with Kafka Consumer

e problem was RowToQuery as you > suggested. The sink was blocking the reads making the Kafka pipeline stall, > due to a misconfiguration of an internal client that is calling an external > service. > > Thanks for your help, > Simone. > > On 16/05/2017 14:01, Kostas Klouda

Re: Problem with Kafka Consumer

Hi Simone, I suppose that you use messageStream.keyBy(…).window(…) right? .windowAll() is not applicable to keyedStreams. Some follow up questions are: In your logs, do you see any error messages? What does your RowToQuery() sink do? Can it be that it blocks and the back pressure makes all

Re: assignTimestampsAndWatermarks not working as expected

Hi Jayesh, Glad that it finally worked! From a first look, I cannot spot anything wrong with the code itself. The only thing I have to note is that the locations of the logs and the printouts you put in your code differ and normally they are not printed in the console. Thanks, Kostas > On

Re: Long running time based Patterns

the within clause. > > Am on 1.3 - expect it'll be fixed by the time stable is out? > > Thanks! > > Moiz > > — > sent from phone > > On 04-May-2017, at 8:12 PM, Kostas Kloudas <k.klou...@data-artisans.com > <mailto:k.klou...@data-artisans.com>> wrot

Re: OperatorState partioning when recovering from failure

Hi Seth, Upon restoring, splits will be re-shuffled among the new tasks, and I believe that state is repartitioned in a round robin way (although I am not 100% sure so I am also including Stefan and Aljoscha in this). The priority queues will be reconstructed based on the restored elements. So

Re: Join two kafka topics

Perfect! Thanks a lot for the clarification! Kostas > On May 4, 2017, at 4:37 PM, Tarek khal wrote: > > Hi Kostas, > > Yes, now is solved by the help of Jason. > > Best, > > > > -- > View this message in context: >

Re: Join two kafka topics

Hi Tarek, This question seems to be a duplicate with your other question “ConnectedStream keyBy issues”, right? I am just asking for clarification. Thanks, Kostas > On May 4, 2017, at 1:41 PM, Tarek khal wrote: > > Hi Aljoscha, > > I tested ConnectedStream and

Re: Long running time based Patterns

Hi Moiz, You are on Flink 1.2 or 1.3? In Flink 1.2 (latest stable) there are no known issues, so this will work correctly. Keep in mind that without any conditions (where-clauses), you will only get all possible 2-tuples of incoming elements, which could also be done with a simple process

Re: CEP timeout occurs even for a successful match when using followedBy

; > Moiz > > On Sat, Apr 29, 2017 at 11:08 PM, Kostas Kloudas <k.klou...@data-artisans.com > <mailto:k.klou...@data-artisans.com>> wrote: > Hi Moiz, > > Here are the instructions on how to build Flink from source: > > https://ci.apache.org/projects/f

Re: CEP timeout occurs even for a successful match when using followedBy

rather use a stable version. > Are there any instructions for building binaries from latest sources? > > Moiz > > On Sat, Apr 29, 2017 at 10:09 PM, Kostas Kloudas <k.klou...@data-artisans.com > <mailto:k.klou...@data-artisans.com>> wrote: > Hi Moiz, > > T

Re: CEP timeout occurs even for a successful match when using followedBy

iz > > On Sat, Apr 29, 2017 at 9:49 PM, Kostas Kloudas <k.klou...@data-artisans.com > <mailto:k.klou...@data-artisans.com>> wrote: > The 1.3 is scheduled for the beginning of June. > > Cheers, > Kostas > >> On Apr 29, 2017, at 6:16 PM, Moiz S Jinia &l

Re: CEP timeout occurs even for a successful match when using followedBy

The 1.3 is scheduled for the beginning of June. Cheers, Kostas > On Apr 29, 2017, at 6:16 PM, Moiz S Jinia wrote: > > Thanks Dawid! > Yes thats what i was expecting. I'll give it a try. > > When do you expect 1.3.0 stable to be out? > > Moiz > > On Sat, Apr 29, 2017

Re: Multiple CEP Patterns

Perfect! And let us know how it goes! Kostas > On Apr 28, 2017, at 5:04 PM, mclendenin wrote: > > Ok, I will try using Flink 1.3 > > > > -- > View this message in context: >

Re: Multiple CEP Patterns

Yes this is the master branch. We have not yet forked the 1.3 branch. And I do not think there is a better way and I am not sure if there can be. Apart from the memory leak that is described in the JIRA, the different NFA’s cannot share any state, so for each one the associated memory

Re: Iterating over keys in state backend

with timestamps smaller than the watermark are processed. Hope this helps, Kostas > On Apr 28, 2017, at 4:08 AM, Ken Krugler <kkrugler_li...@transpac.com> wrote: > > Hi Kostas, > > Thanks for responding. Details in-line below. > >> On Apr 27, 2017, at 1:19am, Kostas Klouda

Re: CEP join across events

you could do something like: > > Pattern. > .begin[Foo]("first") > .where( first => first.baz == 1 ) > .followedBy("next") > .relatedTo("first", { (first, next) => first.bar == next.bar }) > .

Re: Multiple CEP Patterns

28, 2017, at 9:44 AM, Kostas Kloudas <k.klou...@data-artisans.com> > wrote: > > Hi! > > I suppose that by memory errors you mean you run out of memory, right? > > Are you using Flink 1.2 or the current master (upcoming Flink 1.3). > The reason I am asking i

Re: Multiple CEP Patterns

Hi! I suppose that by memory errors you mean you run out of memory, right? Are you using Flink 1.2 or the current master (upcoming Flink 1.3). The reason I am asking is because Flink 1.2 suffered from this https://issues.apache.org/jira/browse/FLINK-5174

Re: CEP join across events

a enumerator of one, which > is the default. > > > On Wed, Apr 26, 2017 at 2:15 AM, Kostas Kloudas <k.klou...@data-artisans.com > <mailto:k.klou...@data-artisans.com>> wrote: > Hi Elias, > > If I understand correctly your use case, you want for an input: >

Re: Iterating over keys in state backend

Hi Ken, Unfortunately, iterating over all keys is not currently supported. Do you have your own custom operator (because you mention “from within the operator…”) or you have a process function (because you mention the “onTimer” method)? Also, could you describe your use case a bit more? You

Re: CEP join across events

Hi Elias, If I understand correctly your use case, you want for an input: event_1 = (type=1, value_a=K, value_b=X) event_2 = (type=2, value_a=K, value_b=X) event_3 = (type=1, value_a=K, value_b=Y) to get a match: event_1, event_2 and discard event_3, right? In this case, Dawid is correct and

Re: Generate Timestamps and emit Watermarks - unordered events - Kafka source

Perfect! Thanks a lot for testing it Luis! And keep us posted if you find anything else. As you may have seen the CEP library is undergoing heavy refactoring for the upcoming release. Kostas > On Apr 25, 2017, at 12:30 PM, Luis Lázaro wrote: > > Hi Aljoscha and Kostas,

Re: Generate Timestamps and emit Watermarks - unordered events - Kafka source

Hi Luis and Aljoscha, In Flink-1.2 late events were not dropped, but they were processed as normal ones. This is fixed for Flink-1.3 with https://issues.apache.org/jira/browse/FLINK-6205 . I would recommend you to switch to the master branch

Re: Window Functions and Empty Panes

that Konstantin provided. There, the example uses a value state to hold the counter, you can do sth similar to keep the flag. Keep in mind that the state will already be scoped by key so you do not have to worry about that either. Kostas > On Apr 18, 2017, at 11:11 PM, Kostas Kloudas <k.klou..

<    1   2   3   4   >