Howdy, ok, so I think agreed on the name (and artifactId) of the two new module: * maven-resolver-transport-jdk (uses Java11+ java.net.http.HttpClient) * maven-resolver-transport-jetty (uses Jetty HttpClient, probably 12.x)
and the existing ones are: * maven-resolver-transport-classpath (CP transport, is not HTTP, just FTR) * maven-resolver-transport-file (file transport, is not HTTP, just FTR) * maven-resolver-transport-http (uses Apache HttpClient 4.x, HTTP/1.1 capable) Cannot rename the misleading "-http" one, as that would cause disruption with existing code, we have to live with it for now (to be renamed in resolver 3.0 or so) :( As for CLI names and Maven4 inclusion in distro (these are Maven4 changes, not resolver changes), I'd propose: * "jdk" included in distro * "jetty" not included in distro (users can add it via .mvn/extensions.xml) * "apache" (and deprecated CLI synonym "native") included in distro * maven-resolver-transport-wagon is to be dropped from Maven4 (not shipped by default, users can add it as core extension and make it work via standard resolver priority mechanism. Wagon itself IS present in the Maven4 core, as it is required for Maven3 compatibility provided by Maven4). Transport priorities (how Resolver selects transport if multiple one exists for same protocol): wagon = -1.0f (unchanged, same as today) https://github.com/apache/maven-resolver/blob/maven-resolver-1.9.16/maven-resolver-transport-wagon/src/main/java/org/eclipse/aether/transport/wagon/WagonTransporterFactory.java#L47 apache = 5.0f (unchanged, same as today) https://github.com/apache/maven-resolver/blob/maven-resolver-1.9.16/maven-resolver-transport-http/src/main/java/org/eclipse/aether/transport/http/HttpTransporterFactory.java#L51 jdk = 10.0f (proposed, to prevail "apache" transport, both will be present in distro) jetty = 15.0f (proposed, to prevail "jdk" transport, if present) Note re priorities: Resolver "prioritized components" mechanism was present since day 0 of resolver, and its purpose was to select a given implementation if multiple ones are suited (like in our case, TransportFactory for HTTP URL). Before Maven 3.9, wagon transport was the ONLY HTTP capable transport present on classpath, so selection was trivial. In Maven 3.9 both wagon and apache transports were present in distro, and apache "prevails" by default (due default priorities, while CLI switches tweak priorities). To not introduce changes in this area, the idea is to position jdk transport above apache (hence priority 10, to prevail apache, become default), while jetty priority of 15 (it is not present in distro by default) is simply to make it "if present, it prevails". So to say, to simplify users' lives who took effort to set it as a core extension (and to not have to tweak more than that). Note1: Essentially we can play "same game" as we did with Maven 3.9: we will switch default transport AGAIN IF user uses Java11+ (but this time from apache/native to jdk), providing fallback for users hitting a blocker bug in the form of `-Dmaven.resolver.transport=apache`, and for those still relying on Wagon (ie. using legacy config), they would need two steps a) make wagon transport be in .mvn/extensions.xml and b) use `-Dmaven.resolver.transport=wagon`. I believe this is acceptable, while we are in "alpha" with Maven 4. Of course, this last paragraph is just a proposal, does not have to happen (we can make it in 4.1 or alike). Note2: the "...if user uses Java11+" is nicely handled by Sisu. If Maven4 remains Java8 bytecode level, and the user uses Java8 to run Maven4, Sisu will silently omit jdk transport (as Java8 will be unable to load the Java11 bytecode). If anyone objects, please yell here. Otherwise will continue along these lines. Thanks T On Fri, Oct 13, 2023 at 1:12 PM Romain Manni-Bucau <[email protected]> wrote: > Le ven. 13 oct. 2023 à 11:07, Tamás Cservenák <[email protected]> a > écrit : > > > Howdy, > > > > Maven Central supports HTTP/2 for quite some time. > > Google Mirror of Maven Central in the EU has supported HTTP/3 since a > while > > ago. > > > > And while I agree that HTTP/2 over HTTP/1.1 is not huge diff (it is, for > > example in the sense of used ports, that may be important on farm-sized > CI, > > also there is noticeable perf improvement as well), the real boost will > be > > probably HTTP/3. Sadly, a year or more ago, when I did perf testing, > > Jetty12 was still alpha, but I wonder what HTTP/3 numbers would be today. > > > > Until you protect servers against ddos, so we shouldnt benefit from there > and boost is not really in these protocol but more in nio usage where you > get way faster than current sync whetever protocol you use. > > > > > I agree also: > > - jdk/jre client is clearly way to go forward (and is even > dependency-less, > > so would make Maven distro even lighter, but unlike > wagon-http-lightweight, > > is fully featured) > > - jetty client is "way forward" for those who may want early adoption on > > things like HTTP/3 etc (at the cost of more deps) > > > > > > On Fri, Oct 13, 2023 at 10:50 AM Romain Manni-Bucau < > [email protected] > > > > > wrote: > > > > > Less is better in terms of stack so jre client is the way forward for > me, > > > it is fast, reliable and keeps getting enhancements. > > > > > > On http2/3 I dont see it very relevant cause it still gets a lot of > > issues > > > and should be blacklisted for a while until serious mitigations are set > > up > > > on servers. > > > > > > In terms of style, virtual threads are just a default async style which > > is > > > not mainstream and will not for years. It also foes not prevent > locking, > > in > > > particular with our current codebase so a full rewrite is relevant if > > there > > > is any will and if not becoming lighter is what is the most relevant. > > > > > > All that to say that staying on jre client, keeping an abstractio is > > what > > > I see as relevant. > > > Other impl can be extensions in an extra repo but sound additional > > > maintenance without end user gain from my window. > > > > > > > > > Le ven. 13 oct. 2023 à 10:30, Michael Osipov <[email protected]> a > > > écrit : > > > > > > > I agree that this is a full rewrite, sadly. > > > > > > > > Have you evaluated what degree of logging both JEtty Client and the > JDK > > > > Client offer? I consider the JDK client likely as useless in this > > regard. > > > > Don't know about Jetty. At least AHC saved me a lot of trouble... > > > > > > > > On 2023/10/13 08:23:33 Tamás Cservenák wrote: > > > > > And just add more context about transports: > > > > > > > > > > Currently we have Wagon and AHC 4.x ("native") transport in Maven > > > (since > > > > > 3.9). Numbers show that "native" is far superior over Wagon (I > think > > we > > > > can > > > > > all agree on this). > > > > > > > > > > Sadly, "native" relies on EOL (or soon EOL?) library AHC 4.x (that > is > > > one > > > > > of the best HTTP libraries I used), and this implies we are stuck > > with > > > > > HTTP/1.1 only and (probably, hopefully?) bugfix releases only. > > > > Ironically, > > > > > the moment we provided "superior" HTTP transport in Maven, we got > > stuck > > > > at > > > > > the same time. While there is AHC 5.x (sync, that is not HTTP/2 > > > capable, > > > > > and async, both require heavy rewrite), switching to it requires > > _full > > > > > rewrite_ as mentioned. Plus the "go async" if you want to leverage > > any > > > > new > > > > > protocol. Basically it is like writing resolver transport from > > scratch. > > > > > > > > > > Hence, I think it is more sane to invest our effort into something > > more > > > > > stable, that at the same time gives us (promises us) future > headroom > > as > > > > > well, and IMHO JDK/JRE net.http.HttpClient and (in my experience) > > Jetty > > > > > HttpClient are such things. There is no need for _full rewrite_ > > between > > > > > major versions. > > > > > > > > > > In the longer term, this will actually lower the needed effort to > > > > maintain > > > > > these resolver transports, while at the same time prevent us being > > > > cornered > > > > > again. > > > > > > > > > > On Fri, Oct 13, 2023 at 10:11 AM Tamás Cservenák < > > [email protected]> > > > > > wrote: > > > > > > > > > > > Howdy, > > > > > > > > > > > > Re perf (and this was already discussed about a year ago, please > do > > > not > > > > > > derail discussion): look around > > > > > > https://github.com/apache/maven-resolver/pull/231 and related > > JIRAs, > > > > > > there are the numbers. TL;DR: jetty and jdk clients are > > consistently > > > > > > fastest and are "side by side" (with the benefit of Jetty doing > > > HTTP/3 > > > > as > > > > > > well, but with the downside of huge dep trail), while Wagon is > > > > consistently > > > > > > the slowest. Differences wildly differ but are very notable. > > > > > > > > > > > > Personally, I am not interested in doing a full rewrite of > existing > > > AHC > > > > > > transport (as the expected amount of bugs doing this is pretty > much > > > on > > > > par > > > > > > with doing fully new transport from scratch). I'd rather just > > choose > > > a > > > > > > library with a more stable API, like Jetty is. Moreover, forcing > > > > "async" > > > > > > style in 2023 is something I'd avoid in today's written Java code > > > (esp > > > > with > > > > > > 21 virtual threads). But sure, if you want to do it, go for it > > (this > > > > work > > > > > > would at least move Maven as a project forward, not backward). > > > > > > > > > > > > On Fri, Oct 13, 2023 at 9:52 AM Michael Osipov < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > >> Regardless of the names, the following questions code to my > mind: > > > > > >> > > > > > >> * Does it make sense to put effort into too many clients? > > > > > >> * How many users will actually switch the client? I bet < 10% > > > > > >> > > > > > >> olegk@ and me assessed many times on JIRA that HTTP/2 will have > > > > little > > > > > >> performance benefit for our use case of transport. I'd like to > > see a > > > > move > > > > > >> to Apache HttpClient 5 Async which does everything, but that is > > > work. > > > > > >> Having both AHC async and Jetty HTTP Client is a logical overlap > > and > > > > > >> unnecessary maintenance burden. While I know nothing about the > > Jetty > > > > > >> Client, the AHC (any version) has exceptional logging output > down > > to > > > > bytes > > > > > >> of streams which helps very often to analyze problem with users. > > > > > >> > > > > > >> I personally refuse and close out issue on JIRA where the user > is > > > not > > > > > >> able to present that kind of debugging information. Poking in > the > > > > dark. > > > > > >> > > > > > >> M > > > > > >> > > > > > >> > > > --------------------------------------------------------------------- > > > > > >> To unsubscribe, e-mail: [email protected] > > > > > >> For additional commands, e-mail: [email protected] > > > > > >> > > > > > >> > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > >
