Yes, in fact we use the Aether Resolver to resolve versions. And the message is actually coming from org.eclipse.aether:aether-impl.
But I shall double check. Best regards Jarmoni ________________________________ From: Slawomir Jaranowski <s.jaranow...@gmail.com> Sent: Thursday, May 25, 2023 2:45:46 PM To: Maven Users List <users@maven.apache.org> Cc: Garret Wilson <gar...@globalmentor.com> Subject: Re: Maven Artifact Resolver not seeing latest plugins on Maven Central on my machine Hi, Short hint org.apache.maven.repository.RepositorySystem is not used for resolving by versions plugin It used for repositorySystem.createPluginArtifact, repositorySystem.createDependencyArtifact https://github.com/mojohaus/versions/blob/master/versions-common/src/main/java/org/codehaus/mojo/versions/api/DefaultVersionsHelper.java czw., 25 maj 2023 o 11:23 Tamás Cservenák <ta...@cservenak.net> napisał(a): > Howdy, > > So Garret, sorry for the long response, and exactly for its length I have > to say in advance: this will NOT solve your problem :( > > For readers: context https://github.com/mojohaus/versions/issues/959 > > OTOH, it sheds some light on the "passive aggressive" stance of some Maven > Core developers (me for example), but for this a short history lesson is > needed. > > There was Maven2 (rewrite of Maven1, so no "history" in there), and then > Maven3 happened. Biggest change in Maven3 was the introduction of Resolver > (Mercury, Sonatype Aether, Eclipse Aether, today Maven Resolver), as Maven2 > had "resolving" code scattered all over the place, duplicated, and causing > bugs and maintenance problems. One of the major goals of Maven3 was to > promise "smooth sailing" for Maven2 users, so full backward compatibility > was the goal as well. This is one of the reasons why the maven-compat > module exists today, it contains Maven2 "compatibility layer" (alternate > implementations from Maven2 times), for plugins that still use Maven2 > codebase. On the other hand, this "smooth sailing" for users introduced > quite challenging (technical) issues below the surface for devs. This is > complicated by the fact that neither Maven2 nor Maven3 had no "well defined > API" per se, as it was really just "a bunch of classes and components". In > reality, a very tiny subset of plugins can afford to depend on > maven-plugin-api ONLY. They usually need more, and depend on maven-core > (the implementation), maven-compat (maven2 compat layer), maven-whatever. > Most Maven could do is somehow "signal" the intent to developers in javadoc > or package naming (ie. by placing class into "internal" Java package > denoting "this is private, please do not touch", example of this can be > seen here https://maven.apache.org/resolver/api-compatibility.html). > > For example, what Maven3 was fighting with sword and fire (very eagerly) > was to achieve, that code reaching to local or remote repo does not do > this: > > https://github.com/apache/maven-verifier/blob/maven-verifier-1.8.0/src/main/java/org/apache/maven/it/Verifier.java#L577 > Similar code was scattered (copy pasted) all across the Maven2 and Maven2 > plugin codebase. Instead, it required devs to use resolver APIs (and leave > layout hidden): > > https://github.com/apache/maven-resolver/blob/maven-resolver-1.9.10/maven-resolver-api/src/main/java/org/eclipse/aether/repository/LocalRepositoryManager.java > > Related problem is reaching out to plugin developers. For some reason > (historically I guess, and it simply got into people's reflex), plugins are > usually compiled against the same version of Maven artifacts as the plugin > declared as "maven prerequisite", the minimal supported Maven version by > plugin. For ASF Maven plugins that is 3.2.5. Hence, they compile against > maven-core 3.2.5, maven-compat 3.2.5 etc (in case of ASF plugins). The > 3.2.5 version was released in 2014. Therefore, whatever we deprecate in > Maven 3.8.x, 3.9.x etc goes unnoticed by plugin devs, as they compile > against classes from 2014, there is no deprecations shown in IDEs or during > compilation. At the same time, at runtime with latest Maven versions, we > must provide binary compatibility, as the Maven "swaps out" plugin declared > maven-core 3.2.5 with current maven-core 3.9.2 or so. > > So this begs the question, "what makes a plugin 2.x or 3.x"? From that > above, one may simply say "a plugin that requires maven-compat at runtime > (uses components or code from) is 2.x plugin". So, for plugins requiring > maven-compat at build time (ie. compile scope), answer is trivial: is 2.x > plugin. Things get a bit more complicated, as almost all otherwise 3.x > plugins require maven-compat in test scope, but this is more "technical > issue", the test framework in use requires it (it relies on maven-compat, > while the plugin tested may not). So, maven-compat in the test scope could > be fine. > > And finally, we arrive to Garret's case: it turns out we have a cuckoo egg > in maven-core (not that it was "secret", but I got really surprised when > dug myself into what is happening here): The > maven-core org.apache.maven.repository.RepositorySystem component. Sadly, > it's name is SAME to Resolver RepositorySystem, but that's not the only > problem with this component. The problem lies in fact that maven-core > contains the component interface, while the implementation for this > component is in > maven-compat org.apache.maven.repository.legacy.LegacyRepositorySystem and > from this moment on, we delve into Maven2 compatibility layer that exists > in parallel with Maven Resolver (this is "salvaged" Maven2 code, doing or > redoing mostly same what Maven Resolver does). Simply put, this seemingly > valid component org.apache.maven.repository.RepositorySystem keeps wide > open gates leading to the rabbit hole of Maven2 codebase that lives in > parallel with Maven3. > > Most revealing were Garret debug messages "[DEBUG] Determining update check > for artifact" that Andrzej and I myself were wrong when we both stated "it > comes from resolver". It does not, it comes from here: > > https://github.com/apache/maven/blob/maven-3.9.2/maven-compat/src/main/java/org/apache/maven/repository/legacy/DefaultUpdateCheckManager.java#L76 > > Now, this "update check manager" coexists and works totally independently > from resolver related classes. they are literally "doing the same thing > differently" and work against same local repository: > > https://github.com/apache/maven-resolver/blob/maven-resolver-1.9.10/maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/DefaultUpdateCheckManager.java > > https://github.com/apache/maven-resolver/blob/maven-resolver-1.9.10/maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/DefaultTrackingFileManager.java > > Consequence: maven plugin using > org.apache.maven.repository.RepositorySystem component is a Maven2 plugin. > > Hence, the conclusion is: > * versions-maven-plugin uses org.apache.maven.repository.RepositorySystem > quite extensively (and debug logs contains logs from maven-compat), it is > Maven2 plugin > * org.apache.maven.repository.RepositorySystem uses Maven2 classes to reach > out to local and remote repository > * transport is cast in steel: users of this component are married to Wagon > (see WagonManager in LegacyRepositorySystem implementation) > * OTOHm resolver local repository format did receive some (mild) changes > since Maven 3.0 > * but, I guess these "Maven2 compat" classes were "frozen" (read: > unchanged) since > * I assume total breakage if some advanced resolver feature is used like > "split repository", like https://issues.apache.org/jira/browse/MNG-7663 > (could be tested) > > Consequences for plugins using > org.apache.maven.repository.RepositorySystem: > * whatever new transport (HTTP/2 or whatever) is introduced by resolver, > they will stick with legacy Maven2 Wagon > * whatever new local repository feature is introduced (or used at runtime!) > by users, they will be unaware of it (or even break with it) > * "shared repo" feature (locking) is what comes to my mind, seems > completely circumvented by legacy code, and may explain some issues users > reported > > Disclaimer: all this by just skimming sources, I may be wrong at several > points! > > === > > As for me, what I will do instead, is to add yet-another passive aggressive > warning for plugins injecting this legacy component :) > (that will be missed by plugin developers, as by overwhelming requests, the > upcoming Maven 3.9.3 will have these kinds of validation warnings "toned > down", not displayed by default) > Also created https://issues.apache.org/jira/browse/MNG-7794 > > On parallel note, seems we will not be able to achieve this: > https://gist.github.com/cstamas/b0605a9fad09de4adcbd4444888baa4c > As many Maven3 plugins MAY be actually Maven2 plugins without knowing > it, hence for Maven4 to support Maven3 plugins will still be needed to drag > Maven2 codebase :( > > For me, this issue is an exact picture of the problem Maven project is > struggling with. The Maven ecosystem cannot be fixed ONLY by Maven devs. > > === > > So, Garret, given your local repository is accessed by: > * local repository holds a "state" > * is modified by maven core (does NOT use maven-compat) that uses resolver > * is modified by maven plugins, that may use maven-compat that use "maven2 > compat", like in your case > * is modified by m2e (I guess uses resolver, but again, as complete maven > core is present in it, am unsure) > > I really cannot tell who or what (my guess) corrupted the "update check" > related files. > As it was said before, your best bet is to "nuke" your local repository, as > it is really just a cache and should be considered as such. > > HTH > T > > > > On Thu, May 25, 2023 at 9:01 AM Karl Heinz Marbaise <khmarba...@gmx.de> > wrote: > > > Hi, > > > > as I wrote on SO... are you in a corporate environment and using a > > repository manager ? > > > > Kind regards > > Karl Heinz Marbaise > > On 24.05.23 18:04, Garret Wilson wrote: > > > I'm writing to this list on the advice of Andrzej Jarmoniuk on > [Versions > > > Maven Plugin Issue > > > #959](https://github.com/mojohaus/versions/issues/959). I have also > > > opened a [Stack Overflow question]( > https://stackoverflow.com/q/76307809) > > > with a bounty, but so far there have been no responses. > > > > > > In short Maven Artifact Resolver on my machine seems to be stuck at > some > > > previous point in time; it does not see the latest versions on Maven > > > Central when I am requested updated plugin versions using Versions > Maven > > > Plugin. It shows that there are newer versions available, but the ones > > > it shows are not the latest available. Before deleting my entire > > > `C:\Users\user\.m2\repository\` directory tree I would prefer to know > > > what is caused this scenario so that it won't happen again in the > > > future. But at the moment I don't even understand what condition (e.g. > > > incorrect timestamps or whatever) is currently causing this behavior. > > > > > > I am using Maven 3.9.1 on Windows 10. I also use Eclipse EE 2023-03, > > > which contains m2e (Eclipse's support for Maven). I start with [this > > > `pom.xml`]( > > > https://github.com/globalmentor/globalmentor-root/blob/bce5bdbac7797b5b9114a72e5da2f4d76f3e24a7/pom.xml > ), > > which uses `org.codehaus.mojo:versions-maven-plugin:2.12.0`, which in > turn > > (I am told) uses Maven Artifact Resolver. (Note that I've tried the > latest > > `org.codehaus.mojo:versions-maven-plugin:2.15.0` as well, with the same > > results. I'm using this POM because it's available online and does not > > contain any version ignores to cause confusion.) > > > > > > I wanted to see what plugins were out of date, so I ran: > > > > > > ```bash > > > mvn versions:display-plugin-updates > > > ``` > > > > > > It shows this: > > > > > > ``` > > > [INFO] The following plugin updates are available: > > > [INFO] maven-failsafe-plugin .......................... 2.22.2 -> > > > 3.0.0-M7 > > > [INFO] maven-release-plugin ............................ 2.5.3 -> > > > 3.0.0-M6 > > > [INFO] maven-site-plugin .............................. 3.12.1 -> > > > 4.0.0-M3 > > > [INFO] maven-surefire-plugin .......................... 2.22.2 -> > > > 3.0.0-M7 > > > [INFO] org.springframework.boot:spring-boot-maven-plugin .. 2.7.3 -> > > > 3.0.5 > > > ``` > > > > > > However in Versions Maven Plugin Issue #959 (see link above), Andrzej > > > Jarmoniuk ran the same command and came up with different answers. Here > > > are two examples: > > > > > > ``` > > > [INFO] org.springframework.boot:spring-boot-maven-plugin .. 2.7.3 -> > > > 3.1.0 > > > ``` > > > > > > Note that my output is only showing v3.0.5 is available for > > > `org.springframework.boot:spring-boot-maven-plugin`. Furthermore there > > > are later versions available for some of the other plugins as well. > > > > > > ``` > > > [INFO] com.akathist.maven.plugins.launch4j:launch4j-maven-plugin 2.1.3 > > > -> 2.4.1 > > > ``` > > > > > > My output doesn't even show > > > `com.akathist.maven.plugins.launch4j:launch4j-maven-plugin`; apparently > > > it thinks thje v2.1.3 listed in the POM is the latest available! > > > > > > It would appear that Maven Artifact Resolver is somehow "stuck" at some > > > earlier point in time on my machine. > > > > > > I ran Maven with the `-X` option, and here is part of the output > related > > > to `com.akathist.maven.plugins.launch4j:launch4j-maven-plugin`: > > > > > > ``` > > > … > > > [DEBUG] Checking > > > com.akathist.maven.plugins.launch4j:launch4j-maven-plugin for updates > > > newer than 2.1.3 > > > [DEBUG] Could not find metadata > > > > > > com.akathist.maven.plugins.launch4j:launch4j-maven-plugin/maven-metadata.xml > > in local (C:\Users\user\.m2\repository) > > > [DEBUG] Skipped remote request for > > > > > > com.akathist.maven.plugins.launch4j:launch4j-maven-plugin/maven-metadata.xml, > > locally cached metadata up-to-date > > > [DEBUG] > > > > [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].version=2.1.3 > > > [DEBUG] > > > > > > [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].artifactVersion=2.1.2 > > > [DEBUG] > > > > > > [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].effectiveVersion=2.1.3 > > > [DEBUG] > > > > > > [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].specified=true > > > … > > > ``` > > > > > > This debug information seems to be saying that it can't find > > > > > > `C:\Users\user\.m2\repository\com\akathist\maven\plugins\launch4j\launch4j-maven-plugin\maven-metadata.xml`. > > And in fact that file does not exist! Instead I have > > > `C:\Users\user\.m2\repository\com\akathist\maven\plugins\launch4j\launch4j-maven-plugin\maven-metadata-central.xml`. > > (I don't know what the differences are.) > > > > > > The more ominous line is this one: > > > > > > > `[DEBUG] Skipped remote request for > > > > > > com.akathist.maven.plugins.launch4j:launch4j-maven-plugin/maven-metadata.xml, > > locally cached metadata up-to-date` > > > > > > What might be causing Maven Resolver on my machine to get "stuck" at an > > > earlier point in time, and/or to skip checking Maven Central altogether > > > for newer versions of many plugins? > > > > > > Garret Wilson > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > > For additional commands, e-mail: users-h...@maven.apache.org > > > > > -- Sławomir Jaranowski
