[GitHub] flink pull request: [FLINK-3322] MemoryManager creates too much GC...

2016-03-09 Thread Xazax-hun
Github user Xazax-hun commented on the pull request: https://github.com/apache/flink/pull/1769#issuecomment-194419456 I think the soft references solution is not worth investigating, and I agree that the best way to solve the problem is to make the operators smarter for the iterative

[GitHub] flink pull request: [FLINK-3322] MemoryManager creates too much GC...

2016-03-09 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/1769#issuecomment-194250033 I think the right way to solve that would actually not be in the MemoryManager (to cache), but in the operators that know that they will need the memory again and ag

[GitHub] flink pull request: [FLINK-3322] MemoryManager creates too much GC...

2016-03-07 Thread ggevay
Github user ggevay commented on the pull request: https://github.com/apache/flink/pull/1769#issuecomment-193635557 The Memory tab in Java Mission Control can probably help with investigating this. --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] flink pull request: [FLINK-3322] MemoryManager creates too much GC...

2016-03-07 Thread Xazax-hun
Github user Xazax-hun commented on the pull request: https://github.com/apache/flink/pull/1769#issuecomment-193443590 > I would be curious about the soft reference implementation as in DetaIteration cases I think it is a valid situation that the job needs less and less memory. Please

[GitHub] flink pull request: [FLINK-3322] MemoryManager creates too much GC...

2016-03-06 Thread ggevay
Github user ggevay commented on the pull request: https://github.com/apache/flink/pull/1769#issuecomment-192962635 > I would be curious about the soft reference implementation as in DetaIteration cases I think it is a valid situation that the job needs less and less memory. A

[GitHub] flink pull request: [FLINK-3322] MemoryManager creates too much GC...

2016-03-06 Thread mbalassi
Github user mbalassi commented on the pull request: https://github.com/apache/flink/pull/1769#issuecomment-192954621 I would be curious about the soft reference implementation as in DetaIteration cases I think it is a valid situation that the job needs less and less memory. Please add

[GitHub] flink pull request: [FLINK-3322] MemoryManager creates too much GC...

2016-03-06 Thread Xazax-hun
GitHub user Xazax-hun opened a pull request: https://github.com/apache/flink/pull/1769 [FLINK-3322] MemoryManager creates too much GC pressure with iterative jobs. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Xazax-hun/flink M