Re: Flink Runtime Exception

2015-06-19 Thread Till Rohrmann
I think Andra wrote that there is *no deadlock*. On Fri, Jun 19, 2015 at 3:18 PM Fabian Hueske fhue...@gmail.com http://mailto:fhue...@gmail.com wrote: Hi Andra, The system should never deadlock. There is a bug somewhere if that happens. Can you check if the program is really stuck?

Re: Flink Runtime Exception

2015-06-19 Thread Fabian Hueske
woops, sorry! Whenever I read the word deadlock I getting a bit nervous and distracted ;-) 2015-06-19 15:21 GMT+02:00 Till Rohrmann trohrm...@apache.org: I think Andra wrote that there is *no deadlock*. On Fri, Jun 19, 2015 at 3:18 PM Fabian Hueske fhue...@gmail.com

Re: Flink Runtime Exception

2015-06-19 Thread Ufuk Celebi
On 19 Jun 2015, at 14:53, Andra Lungu lungu.an...@gmail.com wrote: Another problem that I encountered during the same set of experiments(sorry if I am asking too many questions, I am eager to get things fixed): - for the same configuration, a piece of code runs perfectly on 10GB of input,

Re: Flink Runtime Exception

2015-06-19 Thread Till Rohrmann
Hi Andra, the problem seems to be that the deployment of some tasks takes longer than 100s. From the stack trace it looks as if you're not using the latest master. We had problems with previous version where the deployment call waited for the TM to completely download the user code jars. For

Re: Flink Runtime Exception

2015-06-19 Thread Andra Lungu
Yes, so I am using flink-0.9.0-milestone-1. Was it a problem for this version? I'll just fetch the latest master if this is the case. On Fri, Jun 19, 2015 at 2:12 PM, Till Rohrmann trohrm...@apache.org wrote: Hi Andra, the problem seems to be that the deployment of some tasks takes longer

Re: Flink Runtime Exception

2015-06-19 Thread Till Rohrmann
Yes, it was an issue for the milestone release. On Fri, Jun 19, 2015 at 2:18 PM Andra Lungu lungu.an...@gmail.com wrote: Yes, so I am using flink-0.9.0-milestone-1. Was it a problem for this version? I'll just fetch the latest master if this is the case. On Fri, Jun 19, 2015 at 2:12 PM,

Re: Flink Runtime Exception

2015-06-19 Thread Asterios Katsifodimos
Hi Andra, I would try increasing the memory per task manager, i.e. on a machine with 8 CPUs and 16GBs of memory, instead of spawning 8 TMs with 2GB each, I would try to spawn 2 TMs of 8GBs each. This might help with the spilling problem (in case that the CPU is not your bottleneck, this might