Hi I found ticket in jira: https://issues.apache.org/jira/browse/FLINK-3952
For update to latest 4.0 I can create separate ticket and pull request in nearest 2 day. (I test in my env and all work correctly) Update on 4.1 not so easy: 1) we use tv.cntt:netty-router:jar:1.10 for rest endpoints 2) netty-router mostly stoped development and have support only netty 4.0 3) latest versions say then they support 4.1 netty, but from my opinion they under heavy refactoring and I can't recommend replace netty-router 1.10 on this version we have 2 ways to resolve this problem: 1) remove dependency on netty-router 2) rebuild and repack netty-router 1.10 with new version on netty 4.1 (as PoC I already did this work, and all work correctly, only small changes required) We already have custom akka build and build also netty-router under discussion. Thanks Alexey Diomin 2017-06-08 0:13 GMT+04:00 Greg Hogan <c...@greghogan.com>: > Hi Alexey, > > Are you looking to create pull requests for upgrading Netty 4.0 and/or 4.1? > > Greg > > On Thu, May 18, 2017 at 4:41 AM, Alexey Demin <diomi...@gmail.com> wrote: > > > Hi > > > > Problem not directly in flink, but it you use flink with beam then in > > classpath you have original netty 4.0.27 from flink and netty 4.1.x from > > beam (grpc use netty 4.1.x for communication). > > > > Other interest (specific for me now): netty have custom wrapper for > openssl > > library which have more productivity versus default jdk version, when you > > have enabled ssl cluster communication it's will be very usefull give > users > > select implementation of ssl (default,openssl,boringssl). > > > > But netty have a lot of fixes for openssl/boringssl in latest versions > and > > more preferable do update netty as first step and enable select sslengine > > as second step, not all in one step. > > > > > However, if we do the change at the beginning of the next release > cycle, > > we > > > might have enough exposure time to verify whether things work or not. > > > > We just start 1.4 iteration and have time for testing. > > > > Thank, > > Alexey Demin > > > > > > 2017-05-18 11:48 GMT+04:00 Till Rohrmann <trohrm...@apache.org>: > > > > > Hi Alexey, > > > > > > thanks for looking into it. Are we currently facing any problems with > > Netty > > > 4.0.27 (bugs or performance)? I agree that in general we should try to > > use > > > the latest bug fix release. However, in the past we have seen that they > > > might entail some slight behaviour changes which breaks things on the > > Flink > > > side. Since Netty is quite crucial for Flink, I would be extra careful > > here > > > when bumping versions, especially if there is no strong need for it. > > > > > > However, if we do the change at the beginning of the next release > cycle, > > we > > > might have enough exposure time to verify whether things work or not. > > > > > > Cheers, > > > Till > > > > > > On Thu, May 18, 2017 at 8:51 AM, Alexey Demin <diomi...@gmail.com> > > wrote: > > > > > > > Hi > > > > > > > > For now we use very old netty version. > > > > > > > > Netty 4.0.27.Final released on 02-Apr-15 > > > > > > > > <!-- Don't upgrade for now. Netty versions >= 4.0.28.Final > > > > contain an improvement by Netty, which slices a Netty buffer > > > > instead of doing a memory copy [1] in the > > > > LengthFieldBasedFrameDecoder. In some situations, this > > > > interacts badly with our Netty pipeline leading to OutOfMemory > > > > errors. > > > > > > > > [1] https://github.com/netty/netty/issues/3704 --> > > > > > > > > If we so worry about slice in LengthFieldBasedFrameDecoder we can add > > > > custom > > > > LengthFieldBasedCopyFrameDecoder which extend original > > > > LengthFieldBasedFrameDecoder and override extractFrame for keep > current > > > > behavior. > > > > > > > > With this small changes we can update for last 4.0.x. > > > > > > > > For now LengthFieldBasedFrameDecoder also used in KvStateClient and > > > > KvStateServer. > > > > Can we keep use original LengthFieldBasedFrameDecoder or must also > > change > > > > on LengthFieldBasedCopyFrameDecoder? > > > > > > > > If we want we can migrate on 4.1. > > > > I already did tests and all work correctly, small changes in > > > > NettyBufferPool.java and ChunkedByteBuf.java is required (implement > new > > > > method which added to interface). > > > > > > > > > > > > Thanks > > > > Alexey Diomin > > > > > > > > > >
