Re: TaskManager crashes with PageRank algorithm in Gelly

2018-03-15 Thread Greg Hogan
Termination of the TaskManager by the Linux OOM killer indicates an overallocation of memory and you have set "taskmanager.heap.mb: 139264” on machines with 136 GB. Even though you were able to (temporarily?) resolve the issue by enabling preallocation, you may see degraded performance if syste

Re: TaskManager crashes with PageRank algorithm in Gelly

2018-02-26 Thread Nico Kruber
Hi, without knowing Gelly here, maybe it has to do something with cleaning up the allocated memory as mentioned in [1]: taskmanager.memory.preallocate: Can be either of true or false. Specifies whether task managers should allocate all managed memory when starting up. (DEFAULT: false). When taskma

Re: TaskManager crashes with PageRank algorithm in Gelly

2018-02-22 Thread santoshg
An update - I was able to overcome these issues by setting the preallocate flag to true. Not sure why this fixes the problem. Need to dig a bit deeper. -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

TaskManager crashes with PageRank algorithm in Gelly

2018-02-21 Thread santoshg
Folks, We are running a simple PageRank algorithm in Gelly with about 1M edges and we are seeing that one the TaskManager just crashes. We suspect it is some configuration issue because each TaskManager has a total of 136GB memory and we have 8 of these. So, the total memory is more than enough.