[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-27 Thread Yun Gao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185720#comment-17185720 ] Yun Gao commented on FLINK-18695: - Ok, I will open a PR then.  > Allow NettyBufferPool to allocate heap

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-27 Thread Chesnay Schepler (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185715#comment-17185715 ] Chesnay Schepler commented on FLINK-18695: -- [~gaoyunhaii]  Thank you for testing all of this!

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-27 Thread Yun Gao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185656#comment-17185656 ] Yun Gao commented on FLINK-18695: - I future tested the same job with 1) The current version with netty

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-26 Thread Yun Gao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185027#comment-17185027 ] Yun Gao commented on FLINK-18695: - After more thinks I also agree with that 1) could have least effect

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-26 Thread Chesnay Schepler (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185011#comment-17185011 ] Chesnay Schepler commented on FLINK-18695: -- It would be good to do another test with option 1)

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-26 Thread Zhijiang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17184963#comment-17184963 ] Zhijiang commented on FLINK-18695: -- If I understood correctly, there seems three options to unblock

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-25 Thread Chesnay Schepler (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17184102#comment-17184102 ] Chesnay Schepler commented on FLINK-18695: -- Given that we will allocate a heap memory anyway

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-25 Thread Yun Gao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17184089#comment-17184089 ] Yun Gao commented on FLINK-18695: - Hi [~chesnay] I also agree with you that there are both cases in the

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-25 Thread Chesnay Schepler (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17184076#comment-17184076 ] Chesnay Schepler commented on FLINK-18695: -- [~gaoyunhaii] there are certainly cases where

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-25 Thread Yun Gao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17184054#comment-17184054 ] Yun Gao commented on FLINK-18695: - Hi [~chesnay] I did set the number of heap arenas the same value as

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-25 Thread Chesnay Schepler (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17184035#comment-17184035 ] Chesnay Schepler commented on FLINK-18695: -- Am I assuming correctly that you set the number of

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-25 Thread Yun Gao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17184006#comment-17184006 ] Yun Gao commented on FLINK-18695: - [~chesnay] very sorry for the long delay, I have finished the tests

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-25 Thread Chesnay Schepler (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183849#comment-17183849 ] Chesnay Schepler commented on FLINK-18695: -- [~gaoyunhaii] ping > Allow NettyBufferPool to

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-19 Thread Stephan Ewen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180580#comment-17180580 ] Stephan Ewen commented on FLINK-18695: -- Backporting to older releases the changes that reduce the

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-19 Thread Chesnay Schepler (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180405#comment-17180405 ] Chesnay Schepler commented on FLINK-18695: -- [~gaoyunhaii] Thanks for keeping us updated, these

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-19 Thread Yun Gao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180388#comment-17180388 ] Yun Gao commented on FLINK-18695: - Hi, I have finished the tests with the current master branch, and the

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-16 Thread Yun Gao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178666#comment-17178666 ] Yun Gao commented on FLINK-18695: - I'm currently running tests with the current Netty version to compare

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-16 Thread Chesnay Schepler (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178632#comment-17178632 ] Chesnay Schepler commented on FLINK-18695: -- [~gaoyunhaii] Any insights so far? > Allow

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-08-09 Thread Yun Gao (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17174031#comment-17174031 ] Yun Gao commented on FLINK-18695: - Sorry for the long delay for being in vacation, and I will start

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-07-30 Thread Zhijiang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168391#comment-17168391 ] Zhijiang commented on FLINK-18695: -- {quot} How problematic would it be to backport this to older

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-07-30 Thread Chesnay Schepler (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167861#comment-17167861 ] Chesnay Schepler commented on FLINK-18695: -- [~zjwang] Can you help with these experiments? I

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-07-30 Thread Chesnay Schepler (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167857#comment-17167857 ] Chesnay Schepler commented on FLINK-18695: -- How problematic would it be to backport this to

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-07-29 Thread Zhijiang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167650#comment-17167650 ] Zhijiang commented on FLINK-18695: -- > Do we actually also cover the receiving side and use our own

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-07-29 Thread Chesnay Schepler (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167088#comment-17167088 ] Chesnay Schepler commented on FLINK-18695: -- Allowing netty to use heap buffers is quite easy

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-07-29 Thread Nico Kruber (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17167067#comment-17167067 ] Nico Kruber commented on FLINK-18695: - (unrelated to this thread: [~sewen] Do we actually also cover

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-07-29 Thread Zhijiang (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166914#comment-17166914 ] Zhijiang commented on FLINK-18695: -- I agree with [~sewen]'s above analysis. Regarding the proposed

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-07-29 Thread Ufuk Celebi (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166902#comment-17166902 ] Ufuk Celebi commented on FLINK-18695: - I agree with Stephan that it should not have much of an

[jira] [Commented] (FLINK-18695) Allow NettyBufferPool to allocate heap buffers

2020-07-28 Thread Stephan Ewen (Jira)
[ https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166697#comment-17166697 ] Stephan Ewen commented on FLINK-18695: -- My feeling is that this should be okay in Flink 1.12 (and