Re: No key found restore States

Hi, right now, the way to do it is by using a custom operator, i.e. a OneInputStreamOperator. There you have the low-level control and can set timers based on watermarks or processing time. You can, for example look at StreamMap for a very simple operator or WindowOperator for an operator that

Kafka producer sink message loss?

I am correct in assuming that the Kafka producer sink can lose message? I don't expect exactly-once semantics using Kafka as a sink given Kafka publishing guarantees, but I do expect at least once. I gather from reading the source that the producer is publishing messages asynchronously, as

Re: Event processing time with lateness

Super cool stuff On Fri, Jun 3, 2016 at 10:55 AM, Kostas Kloudas wrote: > You are welcome! > > > On Jun 3, 2016, at 4:40 PM, Igor Berman wrote: > > thanks Kosta > > On 3 June 2016 at 16:47, Kostas Kloudas >

/newbie/ Share state between streams

Hi, I am newbie in Flink and have questions about stream states. I can’t find answers in documentation, but if I just miss one, please link to doc) 1. Is ValueState (and other state classes) are ‘stream' scoped? So that, it is not possible to share same state with two (or more) different

Re: Event processing time with lateness

thanks Kosta On 3 June 2016 at 16:47, Kostas Kloudas wrote: > Hi Igor, > > To handle late events in Flink you would have to implement you own custom > trigger. > > To see a relatively more complex example of such a trigger and how to > implement it, > you can have a

Re: Exception in thread main: No such exception errpr

Hello, Thank you for the advice. I have done that simulation and have successfully got correct results. But there is some problem which I have already raised before. My program basically reads an image file does facial detection (with the help of open cv library) and writes it back to disk. I

Event processing time with lateness

Hi according to presentation of Tyler Akidau https://docs.google.com/presentation/d/13YZy2trPugC8Zr9M8_TfSApSCZBGUDZdzi-WUz95JJw/present Flink supports late arrivals for window processing, while I've seen several question in the userlist regarding late arrivals and answer was - sort of "not for

Re: Writing test for Flink streaming jobs

Thank you for your response. flink-spector looks really nice. I tried but got some errors regarding types, maybe because of the thing Alex mentioned. I am looking forward to the new version. Thanks, Hironori. 2016-05-30 16:45 GMT+09:00 lofifnc : > Hi, > > Flinkspector

Re: S3 as streaming source

Hi all, I think we can use `readFile`, `readFileStream` methods in `StreamExecutionEnvironment` to create streaming source from S3 because data are stored as file in S3. But I haven’t test it. Regards, Chiwan Park > On Jun 3, 2016, at 2:37 PM, Tzu-Li (Gordon) Tai wrote: >

Re: S3 as streaming source

Hi Soumya, No, currently there is no Flink standard supported S3 streaming source. As far as I know, there isn't one out in the public yet either. The community is open to submissions for new connectors, so if you happen to be working on one for S3, you can file up a JIRA to let us know. Also,