Re: Maven Resolver + Wagon
Hi Tamas, this is great news! I agree with you that most protocols are either file:// (project-relative repositories) or https:// based nowadays. Thanks! Am Mi., 15. Dez. 2021 um 14:44 Uhr schrieb Guillaume Nodet : > > Such a massive performance improvement is really nice ! I don't know the > whole history behind wagon and other transports so I don't have a strong > opinion about the location of such an improvement , but I definitely > welcome it ! > > Le mer. 15 déc. 2021 à 09:26, Tamás Cservenák a > écrit : > > > Howdy, > > > > I'd like to pitch a topic about maven-resolver and usage of Wagon in it. > > As IMHO, Wagon use in maven-resolver is far from optimal (is very > > suboptimal). > > > > Typical example is this PR: > > https://github.com/apache/maven-resolver/pull/140 > > > > This PR _halves HTTP requests made by resolver_ against Maven Central (!) > > by utilizing hashes sent by Maven Central in HTTP response header. Hence, > > instead of doing GET /a.jar and then GET /a.jar.sha1 it does both at once > > with one HTTP request. IMHO, this is huge. But, this works only with > > maven-resolver-transport-http that is NOT used with Maven (as it uses > > maven-resolver-transport-wagon). Moreover, doing this in Wagon, that is > > layer by layer ... of abstractions is just very hard. > > > > IMHO, back in Maven2 times, when Wagon was conceived, the "transport > > agnosticism" and "universal transport" was really a life saver: back then, > > people used sticks and duct-tapes to craft "repo solutions", hence access > > like SSHFS, Apache httpd + mod_dav and who knows what were common, and > > Wagon having supporting all these cases was really a cool thing to have. > > > > But fast forward 10+ years, there is really no reason to do this today, as > > the landscape changed a LOT, there are MRMs on every corner popping up like > > mushrooms. I don't really see use of maven-resolver (and maven's use of > > maven-resolver) that does not involve HTTP or FILE. > > > > In short, resolver NOT involving HTTP is something you will VERY RARELY > > see. Or in other words, maven-resolver concerning anything NOT HTTP (and > > FILE) is just sub optimal. > > > > So I propose to retarget maven-resolver (and it's use within maven) to use > > maven-resolver-transport-http instead of Wagon. Wagon, similarly like > > Plexus, is there to stay in Maven, but it's use in maven-resolver is really > > really suboptimal. > > > > > > WDYT? > > > > Tamas > > > > > -- > > Guillaume Nodet - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [VOTE] Release Apache Maven Wrapper version 3.1.0
+1 Le mer. 15 déc. 2021 à 19:38, Jason van zyl a écrit : > +1 > > jvz > > > On Dec 13, 2021, at 4:09 PM, Hervé BOUTEMY > wrote: > > > > Hi, > > > > We solved 18 issues: > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12323721=12350068=Text > > > > Staging repo: > > https://repository.apache.org/content/repositories/maven-1679/ > > > https://repository.apache.org/content/repositories/maven-1679/org/apache/maven/wrapper/maven-wrapper-parent/3.1.0/maven-wrapper-parent-3.1.0-source-release.zip > > > > Source release checksum(s): > > maven-wrapper-parent-3.1.0-source-release.zip sha512: > 9b7362c956cd24e21b46b290e154893ae35fd38431b6fe80f035cd17a97806e9221c71ac14975692f2a948ad25489f5b9d9bbb72441f9ab5eb541da0a86bdef9 > > > > Staging site: > > https://maven.apache.org/wrapper-archives/wrapper-LATEST/ > > > > Guide to testing staged releases: > > https://maven.apache.org/guides/development/guide-testing-releases.html > > > > Vote open for at least 72 hours. > > > > [ ] +1 > > [ ] +0 > > [ ] -1 > > > > > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > For additional commands, e-mail: dev-h...@maven.apache.org > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > -- Arnaud Héritier Twitter/GitHub/... : aheritier
Re: [VOTE] Release Apache Maven Wrapper version 3.1.0
+1 jvz > On Dec 13, 2021, at 4:09 PM, Hervé BOUTEMY wrote: > > Hi, > > We solved 18 issues: > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12323721=12350068=Text > > Staging repo: > https://repository.apache.org/content/repositories/maven-1679/ > https://repository.apache.org/content/repositories/maven-1679/org/apache/maven/wrapper/maven-wrapper-parent/3.1.0/maven-wrapper-parent-3.1.0-source-release.zip > > Source release checksum(s): > maven-wrapper-parent-3.1.0-source-release.zip sha512: > 9b7362c956cd24e21b46b290e154893ae35fd38431b6fe80f035cd17a97806e9221c71ac14975692f2a948ad25489f5b9d9bbb72441f9ab5eb541da0a86bdef9 > > Staging site: > https://maven.apache.org/wrapper-archives/wrapper-LATEST/ > > Guide to testing staged releases: > https://maven.apache.org/guides/development/guide-testing-releases.html > > Vote open for at least 72 hours. > > [ ] +1 > [ ] +0 > [ ] -1 > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Maven Resolver + Wagon
Such a massive performance improvement is really nice ! I don't know the whole history behind wagon and other transports so I don't have a strong opinion about the location of such an improvement , but I definitely welcome it ! Le mer. 15 déc. 2021 à 09:26, Tamás Cservenák a écrit : > Howdy, > > I'd like to pitch a topic about maven-resolver and usage of Wagon in it. > As IMHO, Wagon use in maven-resolver is far from optimal (is very > suboptimal). > > Typical example is this PR: > https://github.com/apache/maven-resolver/pull/140 > > This PR _halves HTTP requests made by resolver_ against Maven Central (!) > by utilizing hashes sent by Maven Central in HTTP response header. Hence, > instead of doing GET /a.jar and then GET /a.jar.sha1 it does both at once > with one HTTP request. IMHO, this is huge. But, this works only with > maven-resolver-transport-http that is NOT used with Maven (as it uses > maven-resolver-transport-wagon). Moreover, doing this in Wagon, that is > layer by layer ... of abstractions is just very hard. > > IMHO, back in Maven2 times, when Wagon was conceived, the "transport > agnosticism" and "universal transport" was really a life saver: back then, > people used sticks and duct-tapes to craft "repo solutions", hence access > like SSHFS, Apache httpd + mod_dav and who knows what were common, and > Wagon having supporting all these cases was really a cool thing to have. > > But fast forward 10+ years, there is really no reason to do this today, as > the landscape changed a LOT, there are MRMs on every corner popping up like > mushrooms. I don't really see use of maven-resolver (and maven's use of > maven-resolver) that does not involve HTTP or FILE. > > In short, resolver NOT involving HTTP is something you will VERY RARELY > see. Or in other words, maven-resolver concerning anything NOT HTTP (and > FILE) is just sub optimal. > > So I propose to retarget maven-resolver (and it's use within maven) to use > maven-resolver-transport-http instead of Wagon. Wagon, similarly like > Plexus, is there to stay in Maven, but it's use in maven-resolver is really > really suboptimal. > > > WDYT? > > Tamas > -- Guillaume Nodet
Re: [DISCUSS] Maven 3.8.x and 3.9.x
Both of those (#622 and #630) look like important changes to merge. Getting the caching extension used and those interfaces exercised by the community is only a good thing. Given there’s only one extension in that area right now, even if the extension point interfaces had to change to accommodate improvements over time it’s not the end of the world. Not sure there’s a convention here for marking interfaces as provisional, but it’s not unforeseeable that you'd want to change the signature of the extension point which will render newer versions of the extension unusable with older versions of Maven. I don’t think there’s anything wrong with that while it’s evolving. I think it’s more important to get the implementation used. If there’s an easy to let users know compatibility constraints are not set in stone yet then all the better. jvz > On Dec 13, 2021, at 5:39 AM, Guillaume Nodet wrote: > > In order to progress on a few issues, I'd like to discuss two points. > * merge https://github.com/apache/maven/pull/628 into the 3.8.x branch > and release 3.8.5 asap. This is a long-standing issue which had a couple > of trial fixes over the past months. > * create a 3.9.x branch to merge https://github.com/apache/maven/pull/630 > and https://github.com/apache/maven/pull/630 > > The 3.9.x will be needed in order to progress on > maven-build-cache-extension. Right now, the tests are done on two branches > that have to be built (for 3.x and 4.x), forbidding any kind of release > (which I don't propose at this time, we first need to extract it to a > separate repo anyway...) > > Thoughts ? > > -- > > Guillaume Nodet - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Maven Resolver + Wagon
Simple test (someone could repeat/verify): build maven with reference build (master) and transport-http w/ smart checksums for central build, focusing on "transfer time" (so always empty local repo). (my) prerequisites: Java11 temurin, Linux, maven 3.8.4 a) build maven master locally (I used 16dd31aba109f5b02269df1ecdebe30f83172f12) = maven-wagon -- this is "reference" b) build locally https://github.com/apache/maven-resolver/pull/140 (it will land as maven-resolver-1.7.3-SNAPSHOT in your local repo) b) build locally https://github.com/apache/maven/pull/635 = maven-http -- this is "contender" For test, I bult maven master (same ref as above) with empty local repo (as it is transfer that is being tested) w/o tests settings.xml: local-repository execution: > rm -R local-repository <== MUST, to nuke local repo! > $LOCAL_BUILD/bin/mvn -V -s settings.xml clean package -DfailIfNoTests=false -Dtest=void -Drat.skip=true And repeat that "execution" step for both contender, for "reference" maven and "contender" maven multiple times (don't forget to nuke local repo!). Locally, I have these times (averages): reference - Total time: 01:44 min contender - Total time: 01:07 min Basically, this shaves off 40 seconds out of transport (+ compile, but it is pretty much same for both) times. HTH Tamas On Wed, Dec 15, 2021 at 9:26 AM Tamás Cservenák wrote: > Howdy, > > I'd like to pitch a topic about maven-resolver and usage of Wagon in it. > As IMHO, Wagon use in maven-resolver is far from optimal (is very > suboptimal). > > Typical example is this PR: > https://github.com/apache/maven-resolver/pull/140 > > This PR _halves HTTP requests made by resolver_ against Maven Central (!) > by utilizing hashes sent by Maven Central in HTTP response header. Hence, > instead of doing GET /a.jar and then GET /a.jar.sha1 it does both at once > with one HTTP request. IMHO, this is huge. But, this works only with > maven-resolver-transport-http that is NOT used with Maven (as it uses > maven-resolver-transport-wagon). Moreover, doing this in Wagon, that is > layer by layer ... of abstractions is just very hard. > > IMHO, back in Maven2 times, when Wagon was conceived, the "transport > agnosticism" and "universal transport" was really a life saver: back then, > people used sticks and duct-tapes to craft "repo solutions", hence access > like SSHFS, Apache httpd + mod_dav and who knows what were common, and > Wagon having supporting all these cases was really a cool thing to have. > > But fast forward 10+ years, there is really no reason to do this today, as > the landscape changed a LOT, there are MRMs on every corner popping up like > mushrooms. I don't really see use of maven-resolver (and maven's use of > maven-resolver) that does not involve HTTP or FILE. > > In short, resolver NOT involving HTTP is something you will VERY RARELY > see. Or in other words, maven-resolver concerning anything NOT HTTP (and > FILE) is just sub optimal. > > So I propose to retarget maven-resolver (and it's use within maven) to use > maven-resolver-transport-http instead of Wagon. Wagon, similarly like > Plexus, is there to stay in Maven, but it's use in maven-resolver is really > really suboptimal. > > > WDYT? > > Tamas > >
Re: [VOTE] Release Apache Maven Wrapper version 3.1.0
+1 On Mon, Dec 13, 2021 at 10:09 PM Hervé BOUTEMY wrote: > Hi, > > We solved 18 issues: > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12323721=12350068=Text > > Staging repo: > https://repository.apache.org/content/repositories/maven-1679/ > > https://repository.apache.org/content/repositories/maven-1679/org/apache/maven/wrapper/maven-wrapper-parent/3.1.0/maven-wrapper-parent-3.1.0-source-release.zip > > Source release checksum(s): > maven-wrapper-parent-3.1.0-source-release.zip sha512: > 9b7362c956cd24e21b46b290e154893ae35fd38431b6fe80f035cd17a97806e9221c71ac14975692f2a948ad25489f5b9d9bbb72441f9ab5eb541da0a86bdef9 > > Staging site: > https://maven.apache.org/wrapper-archives/wrapper-LATEST/ > > Guide to testing staged releases: > https://maven.apache.org/guides/development/guide-testing-releases.html > > Vote open for at least 72 hours. > > [ ] +1 > [ ] +0 > [ ] -1 > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > >
Maven Resolver + Wagon
Howdy, I'd like to pitch a topic about maven-resolver and usage of Wagon in it. As IMHO, Wagon use in maven-resolver is far from optimal (is very suboptimal). Typical example is this PR: https://github.com/apache/maven-resolver/pull/140 This PR _halves HTTP requests made by resolver_ against Maven Central (!) by utilizing hashes sent by Maven Central in HTTP response header. Hence, instead of doing GET /a.jar and then GET /a.jar.sha1 it does both at once with one HTTP request. IMHO, this is huge. But, this works only with maven-resolver-transport-http that is NOT used with Maven (as it uses maven-resolver-transport-wagon). Moreover, doing this in Wagon, that is layer by layer ... of abstractions is just very hard. IMHO, back in Maven2 times, when Wagon was conceived, the "transport agnosticism" and "universal transport" was really a life saver: back then, people used sticks and duct-tapes to craft "repo solutions", hence access like SSHFS, Apache httpd + mod_dav and who knows what were common, and Wagon having supporting all these cases was really a cool thing to have. But fast forward 10+ years, there is really no reason to do this today, as the landscape changed a LOT, there are MRMs on every corner popping up like mushrooms. I don't really see use of maven-resolver (and maven's use of maven-resolver) that does not involve HTTP or FILE. In short, resolver NOT involving HTTP is something you will VERY RARELY see. Or in other words, maven-resolver concerning anything NOT HTTP (and FILE) is just sub optimal. So I propose to retarget maven-resolver (and it's use within maven) to use maven-resolver-transport-http instead of Wagon. Wagon, similarly like Plexus, is there to stay in Maven, but it's use in maven-resolver is really really suboptimal. WDYT? Tamas
