@He Pin, Thanks for bringing this up. So, if I understand correctly, the problem is that there is currently no way to control the underlying allocator exclusively for Pekko. Setting `-Dio.netty.allocator.type=unpooled` would impact Netty's behavior across other parts of the framework.
Does anyone know if this could potentially affect the data exchange network stack in `flink-runtime`, which is also based on Netty? Best, Alex On Sat, 18 Jan 2025 at 04:10, He Pin <he...@apache.org> wrote: > > +1 for Netty4 with UNPOOLED memory allocator to not change the default > > memory footprint. > > That can only be done with another release, otherwise if will reduce the > performance. > > see https://github.com/apache/pekko/pull/1709 > > On 2025/01/17 17:05:06 Maximilian Michels wrote: > > +1 for Netty4 with UNPOOLED memory allocator to not change the default > > memory footprint. > > > > -Max > > > > On Fri, Jan 17, 2025 at 1:15 PM Samrat Deb <decordea...@gmail.com> > wrote: > > > > > > +1 to move to netty4. > > > > > > bests, > > > Samrat > > > > > > On Fri, 17 Jan 2025 at 5:30 PM, Luke Chen <show...@gmail.com> wrote: > > > > > > > Thanks for the summary! > > > > > > > > +1 to upgrade Pekko to have netty 4 in 1.19.2 and 1.20.1 releases. > > > > > > > > Thanks. > > > > Luke > > > > > > > > On Fri, Jan 17, 2025 at 7:50 PM He Pin <he...@apache.org> wrote: > > > > > > > > > +1 to Netty 4 > > > > > > > > > > On 2025/01/16 15:12:40 Alexander Fedulov wrote: > > > > > > Hi all, > > > > > > > > > > > > We have one remaining blocker for the 1.19.2 and 1.20.1 releases, > > > > namely > > > > > > the issue associated with ticket FLINK-36510: *"Upgrade Pekko > from > > > > 1.0.1 > > > > > to > > > > > > 1.1.2"* [1]. Here is the context: > > > > > > > > > > > > - The flink-rpc module is currently based on Pekko 1.0.1, > which > > > > > bundles > > > > > > Netty version 3.10.6. Netty 3.10.6 is the last 3.x release and > > > > > officially > > > > > > reached EOL more than eight years ago. It contains at least > 20 known > > > > > > critical vulnerabilities [2]. > > > > > > - FLINK-36510 [1] upgrades flink-rpc to Pekko 1.1.2, which > > > > introduces > > > > > a > > > > > > long-awaited migration to Netty 4.x. > > > > > > - Memory allocation in Netty 4.x differs from Netty 3.x and > has a > > > > > larger > > > > > > memory footprint with default settings [3]. > > > > > > - Norman Mauerer, Netty's project lead, strongly recommends > moving > > > > > away > > > > > > from Netty 3 as soon as possible [4]. > > > > > > - According to Norman, setting > -Dio.netty.allocator.type=unpooled > > > > > should > > > > > > approximate Netty 3's memory behavior at the expense of > performance > > > > > > improvements that Netty 4 would otherwise provide. That said, > Netty > > > > 4 > > > > > with > > > > > > -Dio.netty.allocator.type=unpooled is not expected to perform > worse > > > > > than > > > > > > Netty 3. > > > > > > - Although this change might seem too substantial for a patch > > > > > release, I > > > > > > propose proceeding with it due to the accumulated risks of > staying > > > > on > > > > > Netty > > > > > > 3.10.6. This will need to be addressed in a 1.20 as a patch > release > > > > > anyway, > > > > > > given that 1.20 is designated as LTS, and we can expect Netty > 3 to > > > > > accrue > > > > > > even more CVEs over time. > > > > > > > > > > > > Here you can find more details of the ongoing discussion [5]. > > > > > > > > > > > > Looking forward to hearing the community's thoughts on whether we > > > > should > > > > > > proceed with the proposed changes. > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-36510 > > > > > > [2] > https://mvnrepository.com/artifact/io.netty/netty/3.10.6.Final > > > > > > [3] > > > > > > > > > > > > > > > > https://issues.apache.org/jira/browse/FLINK-36510?focusedCommentId=17911219&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17911219 > > > > > > [4] > https://github.com/apache/flink/pull/25866#issuecomment-2595168560 > > > > > > [5] https://github.com/apache/flink/pull/25866 > > > > > > > > > > > > Best, > > > > > > Alex > > > > > > > > > > > > > > > > > >
