Re: sharebuffer prune code

Thanks Dawid. I'll rebase against your branch and test it. Would revert back if I hit the issue again. Regards, Shailesh On Sun, May 27, 2018 at 5:54 PM, Dawid Wysakowicz wrote: > The logic for SharedBuffer and in result for prunning will be changed in > FLINK-9418 [1]. We plan to make it

Re: latency critical job

Rong Rong: my flink version is 1.4.2 since we are using the docker env which is sharing disk-io, based on our observation, disk-io spike cased by other process in the same physical machine can lead to long time operator processing. -- Sent from:

Re: latency critical job

Timo: thanks for u suggestion -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: [ANNOUNCE] Apache Flink 1.5.0 release

Congratulations, everyone! On Mon, May 28, 2018 at 1:15 AM, Fabian Hueske wrote: > Thank you Till for serving as a release manager for Flink 1.5! > > 2018-05-25 19:46 GMT+02:00 Till Rohrmann : > > > Quick update: I had to update the date of the release blog post which > also > > changed the

Re: Odd job failure

On Mon, May 28, 2018 at 1:48 AM, Piotr Nowojski wrote: > Most likely suspect is the standard java problem of some dependency > convergence issue. Please check if you are not pulling in multiple Kafka > versions into your class path. Especially your job shouldn’t pull any Kafka > library except

Debugging window processing: can I output window start/end times, prove correctness?

I am learning the tumbling and rolling window API and I was wondering what API calls people use to determine if their events are being assigned to windows as they expect? For example, is there a way to print out the window start and and times for windows as they are being processed, and what

[ANNOUNCE] Final Reminder - Call for Presentations - Flink Forward Berlin 2018

Hi all, This is the final reminder for the call for presentations for Flink Forward Berlin 2018. *The call closes in 7 days* (June 4, 11:59 PM CEST). Submit your talk and get to present your Apache Flink and stream processing ideas, experiences, use cases, and best practices on September 4-5 in

Re: Job execution fails when parallelism is increased beyond 1

I'm using Flink 1.4.2 -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Job execution fails when parallelism is increased beyond 1

Could you tell us which Flink version you are using? On 28.05.2018 14:01, HarshithBolar wrote: I'm submitting a Flink job to a cluster that has three Task Managers via the Flink dashboard. When I set `Parallelism` to 1 (which is default), everything runs as expected. But when I increase

ML in Streaming API

Hello and thanks for the subscription! I am using Streaming API to develop a ML algorithm and i would like your opinions regarding the following issues: 1) The input is read from a big size file with d-dimensional points, and i want to perform a parallel count window. In each parallel count

Re: Writing Table API results to a csv file

Fabian, Jorn: Yes, that was indeed it. When I added the env.execute("MyApp") it worked. Thank you for your help. -Chris On Mon, May 28, 2018 at 5:03 AM, Fabian Hueske wrote: > Hi, > > Jörn is probably right. > > In contrast to print(), which immediately triggers an

Re: Writing Table API results to a csv file

Hi, Jörn is probably right. In contrast to print(), which immediately triggers an execution, writeToSink() just appends a sink operator and requires to explicitly trigger the execution. The INFO messages of the TypeExtractor are "just" telling you, that Row cannot be used as a POJO type, but

Re: Clarification in TumblingProcessing TimeWindow Documentation

I agree, this should be fixed. Thanks for noticing, Dhruv. Would you mind creating a JIRA for this? Thank you, Fabian 2018-05-28 8:39 GMT+02:00 Bowen Li : > Hi Dhruv, > > I can see it's confusing, and it does seem the comment should be improved. > You can find concrete

Re: Large number of sources in Flink Job

Hi Chirag, There have been some issue with very large execution graphs. You might need to adjust the default configuration and configure larger Akka buffers and/or timeouts. Also, 2000 sources means that you run at least 2000 threads at once. The FileInputFormat (and most of its sub-classes) in

Re: Odd job failure

Hi, I think that’s unlikely to happen. As far as I know, the only way to actually unload the classes in JVM is when their class loader is garbage collected, which means all the references in the code to it must vanish. In other words, it should never happen that class is not found while anyone

Re: [ANNOUNCE] Apache Flink 1.5.0 release

Thank you Till for serving as a release manager for Flink 1.5! 2018-05-25 19:46 GMT+02:00 Till Rohrmann : > Quick update: I had to update the date of the release blog post which also > changed the URL. It can now be found here: > >

Re: Clarification in TumblingProcessing TimeWindow Documentation

Hi Dhruv, I can see it's confusing, and it does seem the comment should be improved. You can find concrete explanation of tumbling window and relative arguments at https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/operators/windows.html#tumbling-windows Feel free to open a PR