Hmm, extensions define this kind of lifecycle, at least the afterModelRead which is a hook before it could be immutable no? So not sure how it changes the issue, or do you push to drop extensions support?
Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book <https://www.packtpub.com/application-development/java-ee-8-high-performance> Le mar. 5 janv. 2021 à 11:54, Robert Scholte <rfscho...@apache.org> a écrit : > Currently the model is mutable and this causes issues. > Instead I would like to see that once the BuildPlan is finished, the model > becomes immutable. > > > To give a concrete example: It must be possible for code generating goals > to add dependencies. > Now, when using modello readers/writers you often need to add a required > dependency by hand. > Ideally there will be a hook where a plugin can register additional > dependencies (e.g. dom4j). > > This will make any postprocessing of the pom during build obsolete > > Robert > On 5-1-2021 09:09:11, Matthieu Brouillard <matth...@brouillard.fr> wrote: > > Can you give an example? > Like Romain has shown: dynamically added dependencies or a version > computation. > > > Most important is the support for CI-friendly versions > But if extensions dynamically compute and set the versions in the POM using > the API, the changes will not be reflected. > That's why today one has to use flatten-maven-plugin or any other > plugin/task modifying/dumping the POM model to a pom.xml file and > setting/attaching it the POM. > > If the produced artifacts (consumer pom) were computed from the POM (object > model), every change done by any extension would be part of it. > > > but this happens AFTER using the pom > Not always from the pom.xml. I thought extensions were allowed to provide > ModelLocator & ModelReader to both decide which file to use for a project > and how to build the in memory POM model. > So the truth should not be considered to be in the pom.xml but in the POM. > > > On Mon, Jan 4, 2021 at 10:00 PM Robert Scholte wrote: > > > answers are below. > > > > Robert > > > > On 4-1-2021 16:52:23, Matthieu Brouillard wrote: > > @Robert nothing is broken atm, the changes for consumer/build are > currently > > behind your feature flag. > > Robert Scholte: > > It is active by default, so it is actually not hidden. > > > > > > > > But as I feared previously, and as Romain pointed it, by working at XML > > level (and not at POM level) the produced consumer pom does not reflect > > changes from extensions. > > Robert Scholte: > > Can you give an example? > > > > > > I really thought that all the "consumer/build" stuff would make the > > maven-flatten-plugin useless but it looks like it will not be the case if > > working at XML level. > > Robert Scholte: > > Like with most questions: it depends. Most important is the support for > > CI-friendly versions. In this case you won't need the > flatten-maven-plugin > > anymore. > > However, the plugin can rewrite much more, but this happens AFTER using > > the pom. > > That's something I don't like, because this POM was not used to build the > > project, but it was reassembled afterwards. > > My idea is to provide hooks to parts of the pom that might be adjusted, > > but this is something we can work on during the 4.x releases > > > > Did Romain and I miss the whole point of the "consumer/build" > enhancements > > or is it "just" because current implementation has not yet reached the > > targets/outputs? > > > > On Mon, Jan 4, 2021 at 2:56 PM Romain Manni-Bucau > > wrote: > > > > > Hmm, I don't get a few things of this IT: > > > > > > 1. the formatting seems not expected even if valid (the comments are > > > finishing with the first tag for example "--> > > > public class MyListener extends AbstractMavenLifecycleParticipant { > > > > > > @Override > > > public void afterProjectsRead(final MavenSession session) throws > > > MavenExecutionException { > > > if (session.getCurrentProject() == null) { > > > return; > > > } > > > > > > session.getProjects().forEach(p -> { > > > final Dependency dependency = new Dependency(); > > > dependency.setGroupId("junit"); > > > dependency.setArtifactId("junit"); > > > dependency.setVersion("3.8.1"); > > > p.getDependencies().add(dependency); > > > }); > > > } > > > } > > > > > > > > > 2. If you run mvn (4 snapshot) dependency:tree you get this kind of > > output: > > > > > > [INFO] ------------- > > > >------------- > > > [INFO] Building Multi Chapter Simple Web Application Project > > > 0.9-MNG6957-SNAPSHOT [6/6] > > > [INFO] --------------------------------[ jar > > > ]--------------------------------- > > > [INFO] > > > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @ > > simple-webapp > > > --- > > > [INFO] > > org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT > > > [INFO] +- > > > > > > > > > org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile > > > [INFO] \- junit:junit:jar:3.8.1:compile > > > > > > [INFO] > > > > ------------------------------------------------------------------------ > > > > > > 3. run the build to have produced pom and cat the simple-webapp one: > > > > > > > > > 4.0.0 > > > > > > org.sonatype.mavenbook.multi > > > simple-parent > > > 0.9-MNG6957-SNAPSHOT > > > > > > > > > simple-webapp > > > Multi Chapter Simple Web Application Project > > > > > > > > > org.sonatype.mavenbook.multi > > > simple-weather > > > 0.9-MNG6957-SNAPSHOT > > > > > > > > > > > > simple-webapp > > > > > > > > > > > > org.apache.maven.plugins > > > maven-war-plugin > > > 2.6 > > > > > > > > > > > > > > > > > > > > > As you see the dependency is not there. I guess the expected outout is: > > > > > > > > > > > > 4.0.0 > > > simple-webapp > > > Multi Chapter Simple Web Application Project > > > [description, scm, ..., all central requires sections but not build > ones] > > > > > > > > > org.sonatype.mavenbook.multi > > > simple-weather > > > 0.9-MNG6957-SNAPSHOT > > > > > > > > > junit > > > junit > > > 3.8.1 > > > > > > > > > > > > > > > Am I missing something? > > > > > > Romain Manni-Bucau > > > @rmannibucau | Blog > > > | Old Blog > > > | Github > > > https://github.com/rmannibucau> | > > > LinkedIn | Book > > > > > > > > > https://www.packtpub.com/application-development/java-ee-8-high-performance > > > > > > > > > > > > > Le lun. 4 janv. 2021 à 13:41, Robert Scholte a > > > écrit : > > > > > > > > > > > > > > > > > https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer > > > > is the most complete IT > > > > > > > > On 4-1-2021 12:59:51, Romain Manni-Bucau wrote: > > > > Le lun. 4 janv. 2021 à 12:36, Robert Scholte a > > > > écrit : > > > > > > > > > There's just one thing I want to say: > > > > > I'm having trouble with the term "broken". > > > > > > > > > > > > > Well, literally meant broken as decorelated from the user intent and > > > > extension model. > > > > Anyway, didn't intend to blame but more identify the blockers for a > GA > > so > > > > point was really that it seem that on the two sides only the > producing > > > one > > > > is not yet ready since it keeps does not sanitize the model > completely > > > and > > > > keeps build only data like comments, right? Also not yet clear for me > > if > > > we > > > > loose the extension enrichments there. > > > > > > > > > > > > > If a Maven project could be built with M3.6.3, it can still be > built > > > with > > > > > M4. > > > > > If not, it is either regression (MNG-6957, MNG-7063) which must be > > > fixed, > > > > > or it requires changes to a plugin for understandable reasons > > > > > (maven-pgp-plugin) > > > > > AFAIK an interesting extension like the maven-tiles has been tested > > and > > > > > still works. > > > > > > > > > > > > > Do you have this handy, is it in our test suite? I'd like to check > the > > > > produced pom matches the enriched model but happy to start from > > something > > > > already there. > > > > > > > > > > > > > > > > > > Robert > > > > > > > > > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote: > > > > > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a > > > > > écrit : > > > > > > > > > > > I don't remember all those details anymore, because I hit those > in > > > the > > > > > > beginning. > > > > > > Trying things over and over again I decided that this is probably > > the > > > > > most > > > > > > successful approach. > > > > > > > > > > > > > > > > > > What of the goals was to keep the pom.xml as is as much as > > possible. > > > > > > We can only decide for the specific Maven elements how to handle > > > them, > > > > we > > > > > > should not decide about comments and licenses. > > > > > > BTW, the license issue was hard to solve. You cannot use it from > > the > > > > > pom's > > > > > > , because there might be multiple licenses. > > > > > > > > > > > > > > > > I disagree, it is saner IMO to evolve to support that than doing > > > > > anything else. > > > > > Once again you keep things which don't make sense in a consumed pom > > in > > > > > current impl so i'd say the sucess in a few cases breaks as much > > cases > > > so > > > > > we need to revisit anyway IMHO. > > > > > > > > > > > > > > > > > > > > > > The current implementation is a solid way to ensure we're not > > > breaking > > > > > too > > > > > > much, because Maven controls the XML filters. > > > > > > > > > > > > > > > > Hmm, breaking extensions seems to break too much (I'm not speaking > of > > > > other > > > > > parts which breaks the ecosystem there but just that is sufficient > > IMHO > > > > to > > > > > say we must check back our solution). > > > > > > > > > > > > > > > > > > > > > > Also keep in mind, that I only want Maven to decide which > > > modifications > > > > > > are done. > > > > > > > > > > > > > > > > For the consumed pom I agree but it is consistent with keeping > > > everything > > > > > working instead of breaking too, no? > > > > > > > > > > > > > > > > > > > > > > Current polyglot projects should still work, but they cannot > > benefit > > > > from > > > > > > the build/consumer functions yet. > > > > > > > > > > > > > > > > So pom -> build model is kept, build model -> produced pom is > broken? > > > Is > > > > it > > > > > the complete status? > > > > > Sounds ok for a 4.0 and a 4.1 can fix it if so. > > > > > Just want to ensure first part is not broken at all. > > > > > > > > > > > > > > > > > > > > > > Robert > > > > > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote: > > > > > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a > > > > > > écrit : > > > > > > > > > > > > > > So what I was expecting was: raw xml model -> converted to > > > unified > > > > > > > consumed model -> extensions -> model processing. > > > > > > > > > > > > > > This is only the build pom part. You're missing the consume > part, > > > > where > > > > > > > the xml is distributed. > > > > > > > > > > > > > > > > > > > Yes but with previous chain the consume part is "clean/normalize > -> > > > > dump" > > > > > > since we are using consumed model - only standard model - in > memory > > > > > > already. > > > > > > > > > > > > > > > > > > > Build is raw + enrich, consumer is raw + enrich + reduce > > (removing > > > > > > > relativePath and modules are the first examples, but much more > is > > > > > > possible) > > > > > > > > > > > > > > Going for the in memory was also my first thought, but I would > > > loose > > > > > > > information, hence I came up with the current implementation. > > > > > > > > > > > > > > > > > > > I don't see what you loose ot be honest. > > > > > > You mentionned license but this one is in the pom so not a big > > deal, > > > > > > comments which are undesired IMHO as mentionned and element order > > > which > > > > > can > > > > > > really be discussed since we can desire to enforce an order to > > > > normalize > > > > > > consumption + it shouldn't be important since from the project > > point > > > of > > > > > > view your pom is already "broken"/lost (as all your intelligence > is > > > > lost > > > > > by > > > > > > this "not passthrough" process). > > > > > > So overall I don't see what you would loose from the consumer > side > > > but > > > > I > > > > > > see what you lost from maven ecosystem side. > > > > > > > > > > > > > > > > > > > Again, we're at a point where we can have counter solutions, > but > > > > don't > > > > > > > expect me to implement it. > > > > > > > > > > > > > > > > > > > For now I'm just trying to ensure we agree we don't want to break > > > > > existing > > > > > > extensions and the nice ecosystem we built after years. > > > > > > This was really a move forward and it sounds like we broke it at > > > maven > > > > 4 > > > > > > without any user gain which sounds terrible. > > > > > > > > > > > > > > > > > > > > > > > > > > thanks, > > > > > > > Robert > > > > > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote: > > > > > > > Hi all, > > > > > > > > > > > > > > I kind of join Matthieu thoughts there, there is no point to > work > > > at > > > > > xml > > > > > > > level to create the consumed pom - comments is not a point > since > > it > > > > can > > > > > > > commonly/easily refer to a dropped part of the pom so they > should > > > be > > > > > > > stripped. > > > > > > > Current extension model got proven adapted and adopted, using a > > > lower > > > > > > level > > > > > > > extension API will not since XML is, even if still mainstream, > > > often > > > > > > > replaced by alternative configurations and to have done the > work > > to > > > > > > inject > > > > > > > XML configuration programmatically compred to current option, > it > > > is a > > > > > > pain. > > > > > > > The in memory model should stick to consumed model IMHO - being > > > > > > > programmatic there is no point to make it easier, worse case we > > can > > > > add > > > > > > > helper beans (injectable) but in terms of model it will not > help. > > > > > > > > > > > > > > So what I was expecting was: > > > > > > > > > > > > > > raw xml model -> converted to unified consumed model -> > > extensions > > > -> > > > > > > model > > > > > > > processing. > > > > > > > > > > > > > > Indeed, real chain adds a small processing over the first arrow > > > > (inject > > > > > > > versions for example) but nothing crazy and breaking this > overall > > > > flow > > > > > > > which stays user friendly. > > > > > > > > > > > > > > Strictly speaking the new model is just a built-in extension > for > > me > > > > > which > > > > > > > is particular because it will enforce IDE to integrate a new > > > format - > > > > > > > wheres polyglot extensions or others don't require static > > analyzis > > > by > > > > > > > themself not being "standard". > > > > > > > > > > > > > > That said, there is nothing crazy with current implementation, > it > > > > just > > > > > > > require to be updated to be able to take extension changes into > > > > > account. > > > > > > > This can be done by making the extension model 'spyable' (ie > if a > > > > > > > dependency/plugin is added it will be reflected in the final > > > written > > > > > > > pom.xml). > > > > > > > This sounds - instrumenting the extension model API or doing a > > diff > > > > > after > > > > > > > extension phase - like a compromise and let people gets the > best > > of > > > > > both > > > > > > > worlds to me. > > > > > > > > > > > > > > Wdyt? > > > > > > > > > > > > > > Romain Manni-Bucau > > > > > > > @rmannibucau | Blog > > > > > > > | Old Blog > > > > > > > | Github | > > > > > > > LinkedIn | Book > > > > > > > > > > > > > > > > > > > > > > > > > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a > > > > > > > écrit : > > > > > > > > > > > > > > > Hi Matthieu, > > > > > > > > > > > > > > > > As you understand, something had to be changed to move Maven > > > > forward. > > > > > > > > I've decided to pick up that challenge and came up with the > > > current > > > > > > > > solution. > > > > > > > > > > > > > > > > My main concerns was that I wanted to keep the fileModel as > > much > > > as > > > > > is. > > > > > > > > That includes the license, comments and element order. > > > > > > > > This information if not available in the memory model, so I > > > needed > > > > > the > > > > > > > > original pom file. > > > > > > > > With that in mind, the usage of XMLFilters looks like the > most > > > > > > > appropriate > > > > > > > > solution. > > > > > > > > > > > > > > > > I am certain that XML is still the most used format, so if we > > can > > > > > have > > > > > > > > improvements for those users, I'm already very happy. > > > > > > > > > > > > > > > > And yes, there are plugins that needs to be updated, but > doing > > > > > nothing > > > > > > is > > > > > > > > not an option anymore. > > > > > > > > > > > > > > > > There are more people that share their concerns, but it took > me > > > > > several > > > > > > > > years to reach this point. > > > > > > > > We now have something that seems to work, anybody who can > > improve > > > > or > > > > > > can > > > > > > > > come up with an alternative implementation can do so. > > > > > > > > > > > > > > > > thanks, > > > > > > > > Robert > > > > > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote: > > > > > > > > Thanks Robert for the video link. > > > > > > > > > > > > > > > > I fully understand the rationales behind the separation of > > > > > > > > build/consumer pom and the video provides some insights on it > > and > > > > you > > > > > > > > explain the actual implementation to introduce this change. > > > > > > > > Still I do not fully understand why it was decided to work on > > top > > > > of > > > > > > XML > > > > > > > by > > > > > > > > filtering/enhancing it instead of working at the POM (in > > > > > > > > memory datamodel) level. > > > > > > > > With the current understanding I have, by doing this choice > of > > > > > working > > > > > > at > > > > > > > > XML level, it looks like it was decided to bypass (if not > kill) > > > > core > > > > > > > > extensions that enhance the POM itself and not the pom.xml ; > > > here I > > > > > can > > > > > > > > think of (but probably not limited to): > > > > > > > > - polyglot-maven: do not use XML but other format to describe > > the > > > > POM > > > > > > > > (yaml, json, kotlin, java, other XML formats, ...) > > > > > > > > - jgitver-maven-plugin (or forks like > > > > > maven-git-versioning-extension): > > > > > > > > dynamic computation of projects version based on git history > > > > > > > > > > > > > > > > With the introduction of core extensions, I thought it was a > > move > > > > to > > > > > > open > > > > > > > > the internals and let externals contribute to the > capabilities > > of > > > > > > maven. > > > > > > > > With the move to a XML handling chain, I see it as a > > > > > > restriction/regress > > > > > > > in > > > > > > > > favor of core closed functionalities. An example of that is > > what > > > is > > > > > > > > provided as CIFriendly stuff, IMO it could/should have been > > > > provided > > > > > > by a > > > > > > > > plugin/extension but instead it is hard written in maven core > > and > > > > is > > > > > > not > > > > > > > > opened for external contribution (plugin/extension I mean). > > > > > > > > > > > > > > > > Perhaps I am totally wrong but I think that maven core should > > > > define > > > > > > all > > > > > > > > its expectations at an API level so that extensions/plugins > > could > > > > > hook > > > > > > at > > > > > > > > this API level. The default packaging of maven could/should > > > provide > > > > > > > default > > > > > > > > implementations of those expectations (for example reading a > > > > pom.xml > > > > > > file > > > > > > > > to a POM model, dumping a POM to a pom-4.0.0.xml, > > > > > > transforming/reducing a > > > > > > > > POM to POM-consumer, dumping POM-consumer to > > > > pom-consumer-5.0.0.xml, > > > > > > ...) > > > > > > > > and let extensions/plugins/default implementations work along > > the > > > > > build > > > > > > > > process with the API & POMs to provide different features and > > > > > > > capabilities. > > > > > > > > > > > > > > > > Matthieu > > > > > > > > > > > > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote: > > > > > > > > > > > > > > > > > I've made a recording[1] about it, which hopefully answers > > most > > > > > > > > questions. > > > > > > > > > > > > > > > > > > Robert > > > > > > > > > > > > > > > > > > [1] https://youtu.be/KDAmlNKZJto > > > > > > > > > > > > > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard > > > > > > > > > wrote: > > > > > > > > > > Not exactly sure what work you mean > > > > > > > > > everything related to maven-xml: > > Build/ConsumerPomXMLFilterxxx, > > > > > > > > > Build/ConsumerModelSourcexxxx and the transformer stuff. > > > > > > > > > Especially, when looking at classes like > > CiFriendlyXMLFilter, I > > > > > would > > > > > > > > have > > > > > > > > > thought that such things could have been done elsewhere, > > > working > > > > on > > > > > > the > > > > > > > > > object model (not on the XML stuff) especially for the > > BuildPom > > > > > part. > > > > > > > > > > > > > > > > > > > however specifically the consumer POM integrates with so > > many > > > > > > > external > > > > > > > > > ecosystems > > > > > > > > > We're aligned here, this has to be stable and well defined > > by a > > > > > > schema. > > > > > > > > > > > > > > > > > > Matthieu > > > > > > > > > > > > > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > > > Not exactly sure what work you mean and I fully agree > that > > > > using > > > > > a > > > > > > > core > > > > > > > > > > model should still be the API for plugins and extensions > to > > > > work > > > > > > > with, > > > > > > > > > > however specifically the consumer POM integrates with so > > many > > > > > > > external > > > > > > > > > > ecosystems, I would expect it to be defined in terms of > XML > > > > > Schema > > > > > > > with > > > > > > > > > > explicite semantic (and the inherent compatibility with > > > exiting > > > > > > > POMs). > > > > > > > > > > > > > > > > > > > > Gruss > > > > > > > > > > Bernd > > > > > > > > > > -- > > > > > > > > > > http://bernd.eckenfels.net > > > > > > > > > > ________________________________ > > > > > > > > > > Von: Matthieu BROUILLARD > > > > > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM > > > > > > > > > > An: dev@maven.apache.org > > > > > > > > > > Betreff: maven 4.0.0 new XML stuff > > > > > > > > > > > > > > > > > > > > Hello all, > > > > > > > > > > > > > > > > > > > > regarding the active work occurring for maven 4.0.0 I > > noticed > > > > the > > > > > > > > > > introduction of a lot of new stuff around SAX parsing & > > > > > filtering. > > > > > > > > > > I am wondering if that means that it was decided that the > > > input > > > > > > > format > > > > > > > > of > > > > > > > > > > maven projects will be XML forever meaning probably, > among > > > > > others, > > > > > > > the > > > > > > > > > end > > > > > > > > > > of polyglot extensions. > > > > > > > > > > Could you explain such a move (or point to > > > rationals/documents) > > > > > and > > > > > > > why > > > > > > > > > you > > > > > > > > > > did not leverage working on the in memory object model > > > allowing > > > > > > > > > > extensions/plugins to contribute/hook in the chain of > > > building > > > > > the > > > > > > > > > BuildPOM > > > > > > > > > > & ConsumePOM? In the past I really thought that this move > > to > > > > > 'Build > > > > > > > vs > > > > > > > > > > Consumer' POM would make clear separations between the > > input > > > > > format > > > > > > > of > > > > > > > > > > descriptors and the core system but I perhaps > misunderstood > > > > > things. > > > > > > > > > > > > > > > > > > > > Also, are there plans regarding the future of core > > > extensions? > > > > > > > > > > With core extensions it was possible to hook into the POM > > > model > > > > > > > loading > > > > > > > > > and > > > > > > > > > > do transformations to do dynamic changes but by working > on > > > the > > > > > XML > > > > > > > > > directly > > > > > > > > > > I see a shift (if not red stop) in this > > > contribution/delegation > > > > > > > > > mechanism. > > > > > > > > > > > > > > > > > > > > Thanks for your time & answers. > > > > > > > > > > > > > > > > > > > > Matthieu Brouillard > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
