[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695319#comment-15695319 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631588#comment-15631588 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15631587#comment-15631587 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602443#comment-15602443 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15585571#comment-15585571 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ggevay commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15585514#comment-15585514 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567606#comment-15567606 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526787#comment-15526787 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526077#comment-15526077 ] ASF GitHub Bot commented on FLINK-3322: --- Github user StephanEwen commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522074#comment-15522074 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522071#comment-15522071 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15515452#comment-15515452 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15508776#comment-15508776 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on a diff in the pull

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15508622#comment-15508622 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on a diff in the pull

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505669#comment-15505669 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on a diff in the pull

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505568#comment-15505568 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504633#comment-15504633 ] ASF GitHub Bot commented on FLINK-3322: --- Github user StephanEwen commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503422#comment-15503422 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on a diff in the pull

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503404#comment-15503404 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on a diff in the pull

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503327#comment-15503327 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on a diff in the pull

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503323#comment-15503323 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on a diff in the pull

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503225#comment-15503225 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ggevay commented on a diff in the pull request:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503205#comment-15503205 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ggevay commented on a diff in the pull request:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503178#comment-15503178 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ggevay commented on a diff in the pull request:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503143#comment-15503143 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ggevay commented on a diff in the pull request:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15503042#comment-15503042 ] ASF GitHub Bot commented on FLINK-3322: --- GitHub user ramkrish86 opened a pull request:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502196#comment-15502196 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15498937#comment-15498937 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ggevay commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15498928#comment-15498928 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ggevay commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15493859#comment-15493859 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15493730#comment-15493730 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15489591#comment-15489591 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ramkrish86 commented on the issue:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15487192#comment-15487192 ] ramkrishna.s.vasudevan commented on FLINK-3322: --- [~ggevay] I think I was able to make the

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15486449#comment-15486449 ] ASF GitHub Bot commented on FLINK-3322: --- GitHub user ramkrish86 opened a pull request:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484571#comment-15484571 ] ramkrishna.s.vasudevan commented on FLINK-3322: --- To go forward with this, I think as

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484104#comment-15484104 ] ramkrishna.s.vasudevan commented on FLINK-3322: --- I did try changing the code to ensure that

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15483225#comment-15483225 ] ramkrishna.s.vasudevan commented on FLINK-3322: --- [~ggevay] I agree with you. But in my test

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15482191#comment-15482191 ] Gabor Gevay commented on FLINK-3322: [~ram_krish] You are right, making the sorters resettable will be

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476131#comment-15476131 ] ramkrishna.s.vasudevan commented on FLINK-3322: --- [~ggevay] I once again checked your

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475975#comment-15475975 ] ramkrishna.s.vasudevan commented on FLINK-3322: --- [~ggevay] Thanks for the inputs. Let me see

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15474575#comment-15474575 ] Gabor Gevay commented on FLINK-3322: I've just realized that what I wrote above doesn't cover the

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15474531#comment-15474531 ] Gabor Gevay commented on FLINK-3322: I had an offline chat with [~StephanEwen], and an alternative

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470419#comment-15470419 ] ramkrishna.s.vasudevan commented on FLINK-3322: --- I am able to handle cases for the iterative

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456065#comment-15456065 ] ramkrishna.s.vasudevan commented on FLINK-3322: --- Reading the code I think I need to update

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15438599#comment-15438599 ] Gabor Gevay commented on FLINK-3322: Yes, I think Stephan meant that you would be mainly working on

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15438412#comment-15438412 ] ramkrishna.s.vasudevan commented on FLINK-3322: --- [~StephanEwen] I thought of working on

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437410#comment-15437410 ] Gabor Gevay commented on FLINK-3322: Yes, I volunteer to help :) > MemoryManager creates too much GC

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15437314#comment-15437314 ] Stephan Ewen commented on FLINK-3322: - Would you volunteer to help with this? If yes, I'd be happy to

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436605#comment-15436605 ] Gabor Gevay commented on FLINK-3322: I think it would be very good to finally solve this issue!

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436212#comment-15436212 ] ramkrishna.s.vasudevan commented on FLINK-3322: --- I can work on this and come up with a

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15187493#comment-15187493 ] ASF GitHub Bot commented on FLINK-3322: --- Github user Xazax-hun commented on the pull request:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15186966#comment-15186966 ] ASF GitHub Bot commented on FLINK-3322: --- Github user StephanEwen commented on the pull request:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15183674#comment-15183674 ] ASF GitHub Bot commented on FLINK-3322: --- Github user Xazax-hun commented on the pull request:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182268#comment-15182268 ] ASF GitHub Bot commented on FLINK-3322: --- Github user ggevay commented on the pull request:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182260#comment-15182260 ] ASF GitHub Bot commented on FLINK-3322: --- Github user mbalassi commented on the pull request:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182076#comment-15182076 ] ASF GitHub Bot commented on FLINK-3322: --- GitHub user Xazax-hun opened a pull request:

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168845#comment-15168845 ] Gabor Gevay commented on FLINK-3322: Actually, these are separate runs, so all of them are initial

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168839#comment-15168839 ] Stephan Ewen commented on FLINK-3322: - The numbers look quite similar, only that the initial run

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15167415#comment-15167415 ] Gabor Gevay commented on FLINK-3322: It's making it even worse: false, 500m: 30s false, 5000m: 115s

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15167402#comment-15167402 ] Stephan Ewen commented on FLINK-3322: - How does the off-heap memory setting affect these stats? Do the

[jira] [Commented] (FLINK-3322) MemoryManager creates too much GC pressure with iterative jobs

[ https://issues.apache.org/jira/browse/FLINK-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15167377#comment-15167377 ] Gabor Gevay commented on FLINK-3322: I have constructed a much simpler example to demonstrate the