Threading issue

I am looking to get  readers from kafka / keyBy and Sink working with all 60 threads. For the most part it is working correctly DataStream stream =env.addSource(new FlinkKafkaConsumer08<>("kafkatopic", schema, properties)).setParallelism(60).flatMap(new

Re: Scala vs Java

Sure you can. Examples come with Java and Scala versions. e.g. https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/datastream_api.html On Tue, Mar 21, 2017 at 9:46 AM, Madabhattula Rajesh Kumar < mrajaf...@gmail.com> wrote: > Hi, > > How stable scala API's when compare to Java

Re: ProcessingTimeTimer in ProcessFunction after a savepoint

Thanks, please do keep me posted! > On 20 Mar 2017, at 21:50, Florian König wrote: > > @Aljoscha Thank you for the pointer to ProcessFunction. That looks like a > better approach with less code and other overhead. > > After restoring, the job is both reading new

Re: Job fails to start with S3 savepoint

Not sure if that is the issue here, but the code does a "fs.exists()" for the path. We should not do this anywhere in all checkpointing related code, because this fails frequently with s3 due to its consistency model. This should eventually succeed though, so I think it is not the problem here.

Re: shaded version of legacy kafka connectors

Hi, There currently isn’t any shaded version Kafka 0.8 connector version available, so yes, you would need to do build that yourself. I’m not completely sure if there will be any class name clashes, because the Kafka 0.8 API is typically packaged under `kafka.javaapi.*`, while in 0.9 / 0.10

Re: load balancing of keys to operators

I think it very depends on your use case, maybe you can use combiner first to reduce the amount of records per key. Maybe you can explain your application a bit more (which window, type of aggregations). It often helps e.g. to introduce an artifical key und merge the result of multiple

Re: shaded version of legacy kafka connectors

Hi Gwenhael, I will loop in Gordon becaue he is more familar with the Kafka connectors. Have you experiences with two versions in the same project? Am 20/03/17 um 15:57 schrieb Gwenhael Pasquiers: Hi, Before doing it myself I thought it would be better to ask. We need to consume from