CEP join across events

There doesn't appear to be a way to join events across conditions using the CEP library. Consider events of the form (type, value_a, value_b) on a stream keyed by the value_a field. Under 1.2 you can create a pattern that for a given value_a, as specified by the stream key, there is a match if

回复：Multiple consumers on a subpartition

Hi albert, As I know, if the upstream data will be consumed by multiple consumers, it will generate multiple subpartitions, and each subpartition will correspond to one input channel consumer.So it is one-to-one correspondence among subpartition -> subpartition view -> input channel.

Flink docs in regards to State

Hi, Flink newbie here. I played with the API (built from GitHub master), I encountered some issues but I am not sure if they are limitations or actually by design: 1. the data stream reduce method does not take a RichReduceFunction. The code compiles but throws runtime exception when

Re: put record to kinesis and then trying consume using flink connector

Hi Sathi, I believe the issue is because you pushed the event into the stream and then you started up a consumer app to start reading after that. If you push an event into the kinesis stream prior to starting up a reader that sets its initial stream position to LATEST, it will not read that

Re: put record to kinesis and then trying consume using flink connector

Hi , I also had a question around how long is the data that you broadcast in a stream that is not changing available in operator’s JVM …will it be as long as the operator is alive. What happens when a slot dies. Does the new slot automatically get aware of the broadcasted data? Thanks Sathi

Re: put record to kinesis and then trying consume using flink connector

Hi Gordon, That was a typo, as I was trying to mask off the stream name.. I still had issues with using Latest as the initial stream position , I moved to using AT_TIMESTAMP to solve it, it works fine now. Thanks so much for your response. Sathi From: "Tzu-Li (Gordon) Tai"

Re: Re-keying / sub-keying a stream without repartitioning

Anyone? On Fri, Apr 21, 2017 at 10:15 PM, Elias Levy wrote: > This is something that has come up before on the list, but in a different > context. I have a need to rekey a stream but would prefer the stream to > not be repartitioned. There is no gain to

Re: [BUG?] Cannot Load User Class on Local Environment

I updated the code a little bit for clarity, now the line #56 mentioned in my previous message is line #25. In summary the error I'm getting is this: --- Caused by: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: com.test.Test ClassLoader info: URL

Re: [BUG?] Cannot Load User Class on Local Environment

Hi Stefan, Check the code here: https://gist.github.com/17d82ee7dd921a0d649574a361cc017d , the output is at the bottom of the page. Here are the results of the additional tests you mentioned: 1. I was able to instantiate an inner class (Test$Foo) inside the Ignite closure, no problem with that

Multiple consumers on a subpartition

Hello, Is there a way Flink allow a (pipelined) subpartition to be consumed by multiple consumers? If not, would it make more sense to implement it as multiple input channels for a single subpartition or multiple subpartition views for each input channel? Any suggestion is appreciated. Thanks

Fault tolerance & idempotency on window functions

Hi guys, I have a flink streaming job that reads from kafka, creates some statistics increments and stores this in hbase (using normal puts). I'm using fold function here of with window of few seconds. My tests showed me that restoring state with window functions is not exactly working how I

Re: Generate Timestamps and emit Watermarks - unordered events - Kafka source

Perfect! Thanks a lot for testing it Luis! And keep us posted if you find anything else. As you may have seen the CEP library is undergoing heavy refactoring for the upcoming release. Kostas > On Apr 25, 2017, at 12:30 PM, Luis Lázaro wrote: > > Hi Aljoscha and Kostas,

Re: Problems reading Parquet input from HDFS

Thanks for your suggestions. @Flavio This is very similar to the code I use and yields basically the same problems. The examples are based on flink-1.0-SNAPSHOT and avro-1.7.6. which is more than three years old. Do you have a working setup with newer version of avro and flink? @Jörn I tried

Re: Generate Timestamps and emit Watermarks - unordered events - Kafka source

Hi Aljoscha and Kostas, thanks in advance. Kostas, i followed your recommendation and it seems to be working fine. I did: - upgrade to 1.3.-SNAPSHOT from master branch. - try assign timestamp and emit watermarks using AscendingTimestampExtractor: alerts are correct (do not process late events