Which test cluster to use for checkpointing tests?

Hi all, For testing checkpointing, is it possible to use LocalFlinkMiniCluster? Asking because I’m not seeing checkpoint calls being made to my custom function (implements ListCheckpointed) when I’m running with LocalFlinkMiniCluster. Though I do see entries like this logged: 18/02/23

Re: Machine Learning: Flink and MOA

Hello Christophe, That's very interesting, I've been working with MOA/SAMOA recently and was considering if we could create some easy integration with Flink. I have a Master student this year that could do some work on this, hopefully we can create something interesting there. Regards, Theodore

Re: Is Flink easy to deploy ?

I think you simply missing a bunch of the Flink artifacts. Flink is broken into dozens of pieces, and you need to select from a large set of artifacts what you need to depend on. Typically, there is one Flink artifact per "extension". I ended up with flink-core flink-core

CEP: watermark not generated when using watermark

hi, I was using a user case of cep on flink 1.3.2, as follow: 1 source from kafka configured with 128 partitions 2 data schema: logTime long, impressionId string, orderPlanId long, type int. If two type(click and impression) with the same impressionId and orderPlanId were matched in 30 seconds,

Cannot used managed keyed state in sink

Hi, It seems that I can't used managed keyed state inside sink functions. Is this unsupported with Flink 1.4 or am I doing something wrong ? Regards, Kien ⁣Sent from TypeApp ​

Implementing CountWindow in Window Join and continuous joining for 2 datastreams

Hi , I'm currently working on Flink with a simple stock market analysis.Basically i need to have the sum of 100 element (Count Window)(sliding size : 10) and also sum of 20 element(Count Window) (sliding size: 10) respectively. I realised that i have to calculate the both sum on 2 different

Task manager not able to rejoin job manager after network hicup

We've observed on our flink 1.4.0 setup that if for some reason the networking between the task manager and the job manager gets disrupted then the task manager is never able to reconnect. You'll end up with messages like this getting printed to the log repeatedly Trying to register at

Re: Window with recent messages

Hi Krzysztof, Thanks for sharing your solution! ProcessFunctions are the Swiss army knife of Flink :-) Cheers, Fabian 2018-02-22 19:55 GMT+01:00 Krzysztof Białek : > Hi Fabian, > > Thank you for your suggestion. In the meantime I rethought this problem > and

Re: Timestamp from Kafka record and watermark generation

Thank you very much Aljoscha! 2018-02-23 14:45 GMT+01:00 Aljoscha Krettek : > Hi, > > This is a know issue: https://issues.apache.org/jira/browse/FLINK-8500. > And yes, the workaround is to write an assigner from scratch but you can > start by copying the code of

Re: Is Flink easy to deploy ?

Yes i have looked. For example, if i want to compile and run StreamTableExample.scala from: https://github.com/apache/flink/blob/master/flink-examples/flink-examples-table/src/main/scala/org/apache/flink/table/examples/scala/StreamTableExample.scala I have taken all examples (and also latest

Imports for example ?

I found interesting Scala example from: https://flink.apache.org/news/2017/03/29/table-sql-api-update.html But what imports I should use ? And what in pom.xml and which versions ? BR Esa

Re: Task manager not able to rejoin job manager after network hicup

@Till Is this the expected behaviour or do you suspect something could be going wrong? > On 23. Feb 2018, at 08:59, jelmer wrote: > > We've observed on our flink 1.4.0 setup that if for some reason the > networking between the task manager and the job manager gets

Re: NoClassDefFoundError of a Avro class after cancel then resubmit the same job

@Elias This is a know issue that will be fixed in 1.4.2 which we will do very quickly just because of this bug: https://issues.apache.org/jira/browse/FLINK-8741 . > On 23. Feb 2018, at 05:53, Elias Levy wrote: > >

Re: Timestamp from Kafka record and watermark generation

Hi, This is a know issue: https://issues.apache.org/jira/browse/FLINK-8500 . And yes, the workaround is to write an assigner from scratch but you can start by copying the code of AscendingTimestampExtractor. Sorry for the inconvenience. --

Re: Problem when uploading a java flink program to aws lambda

Could you please post your build setup. Pom file? And maybe also the contents of your Jar file. Best, Aljoscha > On 22. Feb 2018, at 10:39, Kulasangar > wrote: > > I have created a java application using flink api and table api. I can > provide the source

Is Flink easy to deploy ?

I have lot of difficulties to deploy Flink. That is maybe because I am new with Flink and its (Java and Maven) development environment, but I would hear the opinions of others. I would like to use Scala. There are many examples, but often there are missing "imports" and settings in pom.xml. It

Re: Is Flink easy to deploy ?

Have you had a look at the examples? [1] They can be run out of the IDE. Fabian [1] https://github.com/apache/flink/tree/master/flink-examples/flink-examples-streaming/src/main/scala/org/apache/flink/streaming/scala/examples 2018-02-23 13:30 GMT+01:00 Esa Heikkinen

Re: Task manager not able to rejoin job manager after network hicup

We found out there's a taskmanager.exit-on-fatal-akka-error property that will restart flink in this situation but it is not enabled by default and that feels like a rather blunt tool. I expect systems like this to be more resilient to this On 23 February 2018 at 14:42, Aljoscha Krettek