Netty 4 is much better than Netty 3 and we can schedule tests to run for it and make sure the tests all passed for servertimes. Otherwise, we will have a CVEs old netty 3 in classpath.
Flink is using pekko now, and a hard reboot may not be ok in some cases, and because Pekko ships a Netty3 with many CVEs, they may decide to drop pekko and come out with a dedicated rpc implementation based on Netty4 too. But after we migrate to Netty 4, they can defer that, I will ask around the PMC of Flink at here to let them taking a look. 何品 Matthew de Detrich <matthew.dedetr...@aiven.io.invalid> 于2023年9月11日周一 19:35写道： > > that this is a bug, not a feature > > Also meant to say that this is a feature, not a bug > > On Mon, Sep 11, 2023 at 1:34 PM Matthew de Detrich < > matthew.dedetr...@aiven.io> wrote: > > > While I know that people have reservations for keeping the current > > classical transport, > > I just want to relay what I have written at > > https://github.com/apache/incubator-pekko/pull/643 > > i.e. > > > > - Upgrading the classical transport from Netty 3 to Netty 4 was in > context > > quite trivial > > - Dropping classical transport means that we are technically breaking > > semver (we can only do this in Pekko 2.0.x) > > - Not upgrading to Netty3 means that we are shipping code with known > CVE's > > > > For these reasons I personally would prefer to merge the PR and just get > > it done with. Even > > with the PR merged, if we change our minds later nothing is stopping us > > from dropping it > > later (or even reverting the PR when testing is done but I doubt that > will > > occur). > > > > There is of course a risk in that updating to Netty 4 might create some > > regressions > > and hence pekko users would complain but I would then argue that this is > a > > bug, > > not a feature because we then get actual feedback as to how many people > > are using > > the classical transport which allows us to make more comprehensive > > decisions in > > the future. > > > > On Fri, Sep 8, 2023 at 5:56 PM kerr <hepin1...@gmail.com> wrote: > > > >> And I gathered this result on reddit > >> [image: image.png] > >> 何品 > >> > >> > >> kerr <hepin1...@gmail.com> 于2023年9月8日周五 23:51写道： > >> > >>> I just read > >>> > >>> To switch a full cluster restart is required and any overrides for > >>> classic remoting need to be ported to Artery configuration. Artery has > a > >>> completely different protocol, which means that a rolling update is not > >>> supported. > >>> So for Flink use case is that ok? > >>> 何品 > >>> > >>> > >>> kerr <hepin1...@gmail.com> 于2023年9月8日周五 17:10写道： > >>> > >>>> I did a poll on reddit and get the result of > >>>> [image: image.png] > >>>> 何品 > >>>> > >>>> > >>>> kerr <hepin1...@gmail.com> 于2023年8月6日周日 00:33写道： > >>>> > >>>>> > Is it really worth spending any time to support classic remoting? > >>>>> Users > >>>>> would be better served to switch to Artery. > >>>>> > >>>>> I'm one of the user and the China Scala User group may not talk much > >>>>> on web but most of us is using the netty transport, even Lightbend > say the > >>>>> `Artery is the best`. > >>>>> > >>>>> Another online product with large scale nodes(here) is running with > >>>>> Akka 2.4.x with the classic transport, and Flink is using the > netty.tpc > >>>>> transport at least for now too. > >>>>> > >>>>> I think the community can choose when they migrate the nodes to > artery > >>>>> is essential, especially when otherwise your cluster need a hard > reboot. > >>>>> > >>>>> AFAIK, the Artery transport is not wire compatible with the > >>>>> classic one. > >>>>> > >>>>> I suggest: > >>>>> 1. Start to send PRs to open sources projects like Flink with the > help > >>>>> for migrating to the artery transport. > >>>>> 2. Upgrade to the new version of Netty for bugfix and > >>>>> better performance. > >>>>> 3. Suggest all the new community move to the Artery transport. > >>>>> > >>>>> Even Akka was removed it in Akka 2.8.x not Akka 2.7.x. > >>>>> > >>>>> Anyway, different company has different requirements, that just my 2 > >>>>> cents. > >>>>> > >>>>> 何品 > >>>>> > >>>>> > >>>>> PJ Fanning <fannin...@gmail.com> 于2023年8月2日周三 17:29写道： > >>>>> > >>>>>> This will end up in a vote. > >>>>>> > >>>>>> For me, we shipped 1.0.x. Users who don't want big changes can use > >>>>>> that version. We can agree to support 1.0.x with critical fixes. > >>>>>> > >>>>>> We can now start improving the code for the improvement release > >>>>>> (v1.1.x or v2.0.x). Improvements include getting rid of deprecated > >>>>>> code and most of us appear to believe that classic remoting is long > >>>>>> deprecated. > >>>>>> > >>>>>> > >>>>>> On Wed, 2 Aug 2023 at 09:51, Matthew de Detrich > >>>>>> <matthew.dedetr...@aiven.io.invalid> wrote: > >>>>>> > > >>>>>> > > Is it really worth spending any time to support classic > remoting? > >>>>>> Users > >>>>>> > would be better served to switch to Artery. > >>>>>> > > >>>>>> > That depends on what you mean by "support". If agreed on, I would > >>>>>> > propose marking classic remoting as "to be removed" which would > >>>>>> > also mean that the code wouldn't be touched (i.e. no feature > >>>>>> > changes and no bug fixes unless extremely critical). > >>>>>> > > >>>>>> > I am in no way suggesting that we support remoting in a > traditional > >>>>>> sense > >>>>>> > and again the only reason why I am even contemplating this is > >>>>>> because > >>>>>> > of the CVE's. > >>>>>> > > >>>>>> > > I don't think it is likely that a significant portion of Akka > 2.4 > >>>>>> users > >>>>>> > would update to Pekko any time soon if the couldn't update to Akka > >>>>>> 2.5/2.6. > >>>>>> > > >>>>>> > I also had this impression but it's already come up a few times > that > >>>>>> > people still use Akka 2.4.x. They likely haven't bothered updating > >>>>>> > because they didn't see a need to (quite famously Akka is quite > >>>>>> stable > >>>>>> > and there are cases of companies using it for years without > needing > >>>>>> to > >>>>>> > update, as stated by Lightbend's CEO) but with the license change > >>>>>> > it created a catalyst/trigger for some users to consider moving to > >>>>>> Pekko. > >>>>>> > > >>>>>> > On this note I would also be wary in making assumptions on what > >>>>>> versions > >>>>>> > of Akka people happen to be using. As a corollary in the > discussion > >>>>>> on > >>>>>> > dropping JDK 8, when we suggested that we quite quickly got > >>>>>> feedback that > >>>>>> > there are people still using JDK 8. And while I know all of the > >>>>>> arguments > >>>>>> > that you really should not be using JDK 8 can be carried over to > >>>>>> Akka. > >>>>>> > > >>>>>> > Admittedly it's quite hard to get good feedback on what versions > >>>>>> people are > >>>>>> > using, just saying we should be a bit more careful in making these > >>>>>> > assumptions. > >>>>>> > After all, people are still using JDK 8 ;) > >>>>>> > > >>>>>> > > >>>>>> > On Wed, Aug 2, 2023 at 10:19 AM Nicolas Vollmar < > nvoll...@gmail.com> > >>>>>> wrote: > >>>>>> > > >>>>>> > > Akka 2.4 has been EoL since end of 2017. > >>>>>> > > I don't think it is likely that a significant portion of Akka > 2.4 > >>>>>> users > >>>>>> > > would update to Pekko any time soon if the couldn't update to > >>>>>> Akka 2.5/2.6. > >>>>>> > > > >>>>>> > > Is it really worth spending any time to support classic > remoting? > >>>>>> Users > >>>>>> > > would be better served to switch to Artery. > >>>>>> > > > >>>>>> > > On Wed, 2 Aug 2023 at 10:04, Matthew de Detrich > >>>>>> > > <matthew.dedetr...@aiven.io.invalid> wrote: > >>>>>> > > > >>>>>> > > > > If you manage to move to Pekko and were still using classic > >>>>>> remoting, > >>>>>> > > > it's likely you are still on Akka 2.4 or 2.5. If you manage > the > >>>>>> update > >>>>>> > > > to Pekko, going to Artery TCP is a small step. > >>>>>> > > > > >>>>>> > > > One of the reasons why I was saying this is that I was under > the > >>>>>> > > impression > >>>>>> > > > that there are a non-trivial amount of users still on Akka 2.4 > >>>>>> (not sure > >>>>>> > > > why > >>>>>> > > > this is the case but kerr was telling me this). > >>>>>> > > > > >>>>>> > > > > If people are interested in keeping it around and also opt > in > >>>>>> to > >>>>>> > > > maintaining it (which in the first place means, making the > >>>>>> tests for > >>>>>> > > > it work *now*), than fine, but otherwise, being able to use it > >>>>>> on > >>>>>> > > > 1.0.x is already a big benefit. > >>>>>> > > > > >>>>>> > > > Since classic remoting is currently using Netty 3 which has > >>>>>> CVE's > >>>>>> > > > I don't think it's wise to encourage people that still happen > >>>>>> to be > >>>>>> > > > using classic remoting to stay on 1.0.x. If it wasn't for > final > >>>>>> > > > version of netty 3 having CVE's then I would have no qualms > >>>>>> > > > for dropping it in the 1.1.x series (in fact if there was a > >>>>>> > > > hypothetical netty 3 without CVE's we would have already > >>>>>> > > > updated to it in the 1.0.x series and dropped classic remoting > >>>>>> in > >>>>>> > > > 1.1.x without a heartbeat). > >>>>>> > > > > >>>>>> > > > I would propose that if we do accept keeping classic remoting > >>>>>> > > > and updating to Netty 4, we would mark the feature to be > >>>>>> > > > dropped in 1.2.x and aside from that Netty 4 upgrade that > >>>>>> > > > part of code would be untouched unless there are critical > >>>>>> > > > bugs/regressions. > >>>>>> > > > > >>>>>> > > > On Tue, Aug 1, 2023 at 1:38 PM Johannes Rudolph < > >>>>>> > > > johannes.rudo...@gmail.com> > >>>>>> > > > wrote: > >>>>>> > > > > >>>>>> > > > > We definitely should remove it for 1.1.x. There's no > >>>>>> technical reason > >>>>>> > > > > to keep it because the newer artery TCP transport just > >>>>>> supersedes it. > >>>>>> > > > > As discussed before, the problem with keeping old features > >>>>>> around is > >>>>>> > > > > "death by a thousand cuts". The classic remoting backend is > >>>>>> the prime > >>>>>> > > > > example of it because it is basically unmaintained for years > >>>>>> and > >>>>>> > > > > there's basically no expert knowledge available about how to > >>>>>> maintain > >>>>>> > > > > it or how to fix the existing bugs. Testing remoting > backends > >>>>>> has > >>>>>> > > > > shown to be very complicated and often unreliable and being > >>>>>> able to > >>>>>> > > > > remove this particular component from the test matrix will > be > >>>>>> a huge > >>>>>> > > > > improvement. > >>>>>> > > > > > >>>>>> > > > > On Mon, Jul 31, 2023 at 11:28 PM Matthew de Detrich > >>>>>> > > > > <matthew.dedetr...@aiven.io.invalid> wrote: > >>>>>> > > > > > I would prefer adding in netty 4 support for the classic > >>>>>> transport > >>>>>> > > > > > mechanism because it gives > >>>>>> > > > > > people an upgrade path while allowing them to resolve the > >>>>>> CVE issues. > >>>>>> > > > If > >>>>>> > > > > > Pekko 1.1.x didn't > >>>>>> > > > > > support the classic transport mechanism then it forces > >>>>>> Pekko users to > >>>>>> > > > be > >>>>>> > > > > > stuck on 1.0.x > >>>>>> > > > > > with no easy upgrade path. > >>>>>> > > > > > >>>>>> > > > > We are not responsible for giving people an easy upgrade > path > >>>>>> when > >>>>>> > > > > they were stuck on features like classic remoting which has > >>>>>> basically > >>>>>> > > > > been outdated for years. It is very important that, going > >>>>>> forward, > >>>>>> > > > > that we focus our attention on the most important features > >>>>>> and make > >>>>>> > > > > sure to not get stuck in maintaining old parts that we > cannot > >>>>>> (or > >>>>>> > > > > don't want to) care for. > >>>>>> > > > > > >>>>>> > > > > If people are interested in keeping it around and also opt > in > >>>>>> to > >>>>>> > > > > maintaining it (which in the first place means, making the > >>>>>> tests for > >>>>>> > > > > it work *now*), than fine, but otherwise, being able to use > >>>>>> it on > >>>>>> > > > > 1.0.x is already a big benefit. > >>>>>> > > > > > >>>>>> > > > > Apart from that the update path is pretty clear: > >>>>>> > > > > * move to Pekko 1.0.x > >>>>>> > > > > * move to Arterty TCP > >>>>>> > > > > * move to Pekko 1.1.x > >>>>>> > > > > > >>>>>> > > > > If you manage to move to Pekko and were still using classic > >>>>>> remoting, > >>>>>> > > > > it's likely you are still on Akka 2.4 or 2.5. If you manage > >>>>>> the update > >>>>>> > > > > to Pekko, going to Artery TCP is a small step. > >>>>>> > > > > > >>>>>> > > > > Johannes > >>>>>> > > > > > >>>>>> > > > > > >>>>>> > --------------------------------------------------------------------- > >>>>>> > > > > To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > >>>>>> > > > > For additional commands, e-mail: dev-h...@pekko.apache.org > >>>>>> > > > > > >>>>>> > > > > > >>>>>> > > > > >>>>>> > > > -- > >>>>>> > > > > >>>>>> > > > Matthew de Detrich > >>>>>> > > > > >>>>>> > > > *Aiven Deutschland GmbH* > >>>>>> > > > > >>>>>> > > > Immanuelkirchstraße 26, 10405 Berlin > >>>>>> > > > > >>>>>> > > > Amtsgericht Charlottenburg, HRB 209739 B > >>>>>> > > > > >>>>>> > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >>>>>> > > > > >>>>>> > > > *m:* +491603708037 > >>>>>> > > > > >>>>>> > > > *w:* aiven.io *e:* matthew.dedetr...@aiven.io > >>>>>> > > > > >>>>>> > > > >>>>>> > > >>>>>> > > >>>>>> > -- > >>>>>> > > >>>>>> > Matthew de Detrich > >>>>>> > > >>>>>> > *Aiven Deutschland GmbH* > >>>>>> > > >>>>>> > Immanuelkirchstraße 26, 10405 Berlin > >>>>>> > > >>>>>> > Amtsgericht Charlottenburg, HRB 209739 B > >>>>>> > > >>>>>> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >>>>>> > > >>>>>> > *m:* +491603708037 > >>>>>> > > >>>>>> > *w:* aiven.io *e:* matthew.dedetr...@aiven.io > >>>>>> > >>>>>> > --------------------------------------------------------------------- > >>>>>> To unsubscribe, e-mail: dev-unsubscr...@pekko.apache.org > >>>>>> For additional commands, e-mail: dev-h...@pekko.apache.org > >>>>>> > >>>>>> > > > > -- > > > > Matthew de Detrich > > > > *Aiven Deutschland GmbH* > > > > Immanuelkirchstraße 26, 10405 Berlin > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > > > *m:* +491603708037 > > > > *w:* aiven.io *e:* matthew.dedetr...@aiven.io > > > > > -- > > Matthew de Detrich > > *Aiven Deutschland GmbH* > > Immanuelkirchstraße 26, 10405 Berlin > > Amtsgericht Charlottenburg, HRB 209739 B > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > *m:* +491603708037 > > *w:* aiven.io *e:* matthew.dedetr...@aiven.io >
