@Robert nothing is broken atm, the changes for consumer/build are currently
behind your feature flag.
But as I feared previously, and as Romain pointed it, by working at XML
level (and not at POM level) the produced consumer pom does not reflect
changes from extensions.
I really thought that all the "consumer/build" stuff would make the
maven-flatten-plugin useless but it looks like it will not be the case if
working at XML level.
Did Romain and I miss the whole point of the "consumer/build" enhancements
or is it "just" because current implementation has not yet reached the
targets/outputs?
On Mon, Jan 4, 2021 at 2:56 PM Romain Manni-Bucau <rmannibu...@gmail.com>
wrote:
> Hmm, I don't get a few things of this IT:
>
> 1. the formatting seems not expected even if valid (the comments are
> finishing with the first tag for example "--><project xmlns=")
> 2. produced poms keep the parent, build sections (including management one
> and extensions for ex) so it seems it just resolves the versions for the
> parent and dependencies
> 3. until I did something wrong the extension API is really broken (as there
> is a regression which must be fixed) - was why i jumped into this thread,
> for me this is not good at all for our community
>
> About 3, let me explain what i did to see if I missed something:
>
> 1. I created a custom extension with this listener:
>
> @Component(role = AbstractMavenLifecycleParticipant.class, hint =
> "rmannibucau-test")
> public class MyListener extends AbstractMavenLifecycleParticipant {
>
> @Override
> public void afterProjectsRead(final MavenSession session) throws
> MavenExecutionException {
> if (session.getCurrentProject() == null) {
> return;
> }
>
> session.getProjects().forEach(p -> {
> final Dependency dependency = new Dependency();
> dependency.setGroupId("junit");
> dependency.setArtifactId("junit");
> dependency.setVersion("3.8.1");
> p.getDependencies().add(dependency);
> });
> }
> }
>
>
> 2. If you run mvn (4 snapshot) dependency:tree you get this kind of output:
>
> [INFO] -------------< org.sonatype.mavenbook.multi:simple-webapp
> >-------------
> [INFO] Building Multi Chapter Simple Web Application Project
> 0.9-MNG6957-SNAPSHOT [6/6]
> [INFO] --------------------------------[ jar
> ]---------------------------------
> [INFO]
> [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @ simple-webapp
> ---
> [INFO] org.sonatype.mavenbook.multi:simple-webapp:jar:0.9-MNG6957-SNAPSHOT
> [INFO] +-
>
> org.sonatype.mavenbook.multi:simple-weather:jar:0.9-MNG6957-SNAPSHOT:compile
> [INFO] \- junit:junit:jar:3.8.1:compile
> <-- THIS IS WHAT WE WANT TO SEE
> [INFO]
> ------------------------------------------------------------------------
>
> 3. run the build to have produced pom and cat the simple-webapp one:
>
> <?xml version="1.0" encoding="UTF-8"?><!--
> Licensed to the Apache Software Foundation (ASF) under one
> or more contributor license agreements. See the NOTICE file
> distributed with this work for additional information
> regarding copyright ownership. The ASF licenses this file
> to you under the Apache License, Version 2.0 (the
> "License"); you may not use this file except in compliance
> with the License. You may obtain a copy of the License at
>
> http://www.apache.org/licenses/LICENSE-2.0
>
> Unless required by applicable law or agreed to in writing,
> software distributed under the License is distributed on an
> "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
> KIND, either express or implied. See the License for the
> specific language governing permissions and limitations
> under the License.
> --><project xmlns="http://maven.apache.org/POM/4.0.0"; xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="
> http://maven.apache.org/POM/4.0.0
> https://maven.apache.org/xsd/maven-4.0.0.xsd";>
> <modelVersion>4.0.0</modelVersion>
> <parent>
> <groupId>org.sonatype.mavenbook.multi</groupId>
> <artifactId>simple-parent</artifactId>
> <version>0.9-MNG6957-SNAPSHOT</version>
> </parent>
>
> <artifactId>simple-webapp</artifactId>
> <name>Multi Chapter Simple Web Application Project</name>
> <dependencies>
> <dependency>
> <groupId>org.sonatype.mavenbook.multi</groupId>
> <artifactId>simple-weather</artifactId>
> <version>0.9-MNG6957-SNAPSHOT</version>
> </dependency>
> </dependencies>
> <build>
> <finalName>simple-webapp</finalName>
> <pluginManagement>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-war-plugin</artifactId>
> <version>2.6</version>
> </plugin>
> </plugins>
> </pluginManagement>
> </build>
> </project>
>
> As you see the dependency is not there. I guess the expected outout is:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <project xmlns="http://maven.apache.org/POM/4.0.0"; xmlns:xsi="
> http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="
> http://maven.apache.org/POM/4.0.0
> https://maven.apache.org/xsd/maven-4.0.0.xsd";>
> <modelVersion>4.0.0</modelVersion>
> <artifactId>simple-webapp</artifactId>
> <name>Multi Chapter Simple Web Application Project</name>
> [description, scm, ..., all central requires sections but not build ones]
> <dependencies>
> <dependency>
> <groupId>org.sonatype.mavenbook.multi</groupId>
> <artifactId>simple-weather</artifactId>
> <version>0.9-MNG6957-SNAPSHOT</version>
> </dependency>
> <dependency>
> <groupId>junit</groupId>
> <artifactId>junit</artifactId>
> <version>3.8.1</version>
> </dependency>
> </dependencies>
> </project>
>
> Am I missing something?
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> | Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le lun. 4 janv. 2021 à 13:41, Robert Scholte <rfscho...@apache.org> a
> écrit :
>
> >
> >
> https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer
> > is the most complete IT
> >
> > On 4-1-2021 12:59:51, Romain Manni-Bucau <rmannibu...@gmail.com> wrote:
> > Le lun. 4 janv. 2021 à 12:36, Robert Scholte a
> > écrit :
> >
> > > There's just one thing I want to say:
> > > I'm having trouble with the term "broken".
> > >
> >
> > Well, literally meant broken as decorelated from the user intent and
> > extension model.
> > Anyway, didn't intend to blame but more identify the blockers for a GA so
> > point was really that it seem that on the two sides only the producing
> one
> > is not yet ready since it keeps does not sanitize the model completely
> and
> > keeps build only data like comments, right? Also not yet clear for me if
> we
> > loose the extension enrichments there.
> >
> >
> > > If a Maven project could be built with M3.6.3, it can still be built
> with
> > > M4.
> > > If not, it is either regression (MNG-6957, MNG-7063) which must be
> fixed,
> > > or it requires changes to a plugin for understandable reasons
> > > (maven-pgp-plugin)
> > > AFAIK an interesting extension like the maven-tiles has been tested and
> > > still works.
> > >
> >
> > Do you have this handy, is it in our test suite? I'd like to check the
> > produced pom matches the enriched model but happy to start from something
> > already there.
> >
> >
> > >
> > > Robert
> > >
> > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote:
> > > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a
> > > écrit :
> > >
> > > > I don't remember all those details anymore, because I hit those in
> the
> > > > beginning.
> > > > Trying things over and over again I decided that this is probably the
> > > most
> > > > successful approach.
> > > >
> > > >
> > > > What of the goals was to keep the pom.xml as is as much as possible.
> > > > We can only decide for the specific Maven elements how to handle
> them,
> > we
> > > > should not decide about comments and licenses.
> > > > BTW, the license issue was hard to solve. You cannot use it from the
> > > pom's
> > > > , because there might be multiple licenses.
> > > >
> > >
> > > I disagree, it is saner IMO to evolve to support that than doing
> > > anything else.
> > > Once again you keep things which don't make sense in a consumed pom in
> > > current impl so i'd say the sucess in a few cases breaks as much cases
> so
> > > we need to revisit anyway IMHO.
> > >
> > >
> > > >
> > > > The current implementation is a solid way to ensure we're not
> breaking
> > > too
> > > > much, because Maven controls the XML filters.
> > > >
> > >
> > > Hmm, breaking extensions seems to break too much (I'm not speaking of
> > other
> > > parts which breaks the ecosystem there but just that is sufficient IMHO
> > to
> > > say we must check back our solution).
> > >
> > >
> > > >
> > > > Also keep in mind, that I only want Maven to decide which
> modifications
> > > > are done.
> > > >
> > >
> > > For the consumed pom I agree but it is consistent with keeping
> everything
> > > working instead of breaking too, no?
> > >
> > >
> > > >
> > > > Current polyglot projects should still work, but they cannot benefit
> > from
> > > > the build/consumer functions yet.
> > > >
> > >
> > > So pom -> build model is kept, build model -> produced pom is broken?
> Is
> > it
> > > the complete status?
> > > Sounds ok for a 4.0 and a 4.1 can fix it if so.
> > > Just want to ensure first part is not broken at all.
> > >
> > >
> > > >
> > > > Robert
> > > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote:
> > > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a
> > > > écrit :
> > > >
> > > > > > So what I was expecting was: raw xml model -> converted to
> unified
> > > > > consumed model -> extensions -> model processing.
> > > > >
> > > > > This is only the build pom part. You're missing the consume part,
> > where
> > > > > the xml is distributed.
> > > > >
> > > >
> > > > Yes but with previous chain the consume part is "clean/normalize ->
> > dump"
> > > > since we are using consumed model - only standard model - in memory
> > > > already.
> > > >
> > > >
> > > > > Build is raw + enrich, consumer is raw + enrich + reduce (removing
> > > > > relativePath and modules are the first examples, but much more is
> > > > possible)
> > > > >
> > > > > Going for the in memory was also my first thought, but I would
> loose
> > > > > information, hence I came up with the current implementation.
> > > > >
> > > >
> > > > I don't see what you loose ot be honest.
> > > > You mentionned license but this one is in the pom so not a big deal,
> > > > comments which are undesired IMHO as mentionned and element order
> which
> > > can
> > > > really be discussed since we can desire to enforce an order to
> > normalize
> > > > consumption + it shouldn't be important since from the project point
> of
> > > > view your pom is already "broken"/lost (as all your intelligence is
> > lost
> > > by
> > > > this "not passthrough" process).
> > > > So overall I don't see what you would loose from the consumer side
> but
> > I
> > > > see what you lost from maven ecosystem side.
> > > >
> > > >
> > > > > Again, we're at a point where we can have counter solutions, but
> > don't
> > > > > expect me to implement it.
> > > > >
> > > >
> > > > For now I'm just trying to ensure we agree we don't want to break
> > > existing
> > > > extensions and the nice ecosystem we built after years.
> > > > This was really a move forward and it sounds like we broke it at
> maven
> > 4
> > > > without any user gain which sounds terrible.
> > > >
> > > >
> > > > >
> > > > > thanks,
> > > > > Robert
> > > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote:
> > > > > Hi all,
> > > > >
> > > > > I kind of join Matthieu thoughts there, there is no point to work
> at
> > > xml
> > > > > level to create the consumed pom - comments is not a point since it
> > can
> > > > > commonly/easily refer to a dropped part of the pom so they should
> be
> > > > > stripped.
> > > > > Current extension model got proven adapted and adopted, using a
> lower
> > > > level
> > > > > extension API will not since XML is, even if still mainstream,
> often
> > > > > replaced by alternative configurations and to have done the work to
> > > > inject
> > > > > XML configuration programmatically compred to current option, it
> is a
> > > > pain.
> > > > > The in memory model should stick to consumed model IMHO - being
> > > > > programmatic there is no point to make it easier, worse case we can
> > add
> > > > > helper beans (injectable) but in terms of model it will not help.
> > > > >
> > > > > So what I was expecting was:
> > > > >
> > > > > raw xml model -> converted to unified consumed model -> extensions
> ->
> > > > model
> > > > > processing.
> > > > >
> > > > > Indeed, real chain adds a small processing over the first arrow
> > (inject
> > > > > versions for example) but nothing crazy and breaking this overall
> > flow
> > > > > which stays user friendly.
> > > > >
> > > > > Strictly speaking the new model is just a built-in extension for me
> > > which
> > > > > is particular because it will enforce IDE to integrate a new
> format -
> > > > > wheres polyglot extensions or others don't require static analyzis
> by
> > > > > themself not being "standard".
> > > > >
> > > > > That said, there is nothing crazy with current implementation, it
> > just
> > > > > require to be updated to be able to take extension changes into
> > > account.
> > > > > This can be done by making the extension model 'spyable' (ie if a
> > > > > dependency/plugin is added it will be reflected in the final
> written
> > > > > pom.xml).
> > > > > This sounds - instrumenting the extension model API or doing a diff
> > > after
> > > > > extension phase - like a compromise and let people gets the best of
> > > both
> > > > > worlds to me.
> > > > >
> > > > > Wdyt?
> > > > >
> > > > > Romain Manni-Bucau
> > > > > @rmannibucau | Blog
> > > > > | Old Blog
> > > > > | Github |
> > > > > LinkedIn | Book
> > > > >
> > > > >
> > > > >
> > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a
> > > > > écrit :
> > > > >
> > > > > > Hi Matthieu,
> > > > > >
> > > > > > As you understand, something had to be changed to move Maven
> > forward.
> > > > > > I've decided to pick up that challenge and came up with the
> current
> > > > > > solution.
> > > > > >
> > > > > > My main concerns was that I wanted to keep the fileModel as much
> as
> > > is.
> > > > > > That includes the license, comments and element order.
> > > > > > This information if not available in the memory model, so I
> needed
> > > the
> > > > > > original pom file.
> > > > > > With that in mind, the usage of XMLFilters looks like the most
> > > > > appropriate
> > > > > > solution.
> > > > > >
> > > > > > I am certain that XML is still the most used format, so if we can
> > > have
> > > > > > improvements for those users, I'm already very happy.
> > > > > >
> > > > > > And yes, there are plugins that needs to be updated, but doing
> > > nothing
> > > > is
> > > > > > not an option anymore.
> > > > > >
> > > > > > There are more people that share their concerns, but it took me
> > > several
> > > > > > years to reach this point.
> > > > > > We now have something that seems to work, anybody who can improve
> > or
> > > > can
> > > > > > come up with an alternative implementation can do so.
> > > > > >
> > > > > > thanks,
> > > > > > Robert
> > > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote:
> > > > > > Thanks Robert for the video link.
> > > > > >
> > > > > > I fully understand the rationales behind the separation of
> > > > > > build/consumer pom and the video provides some insights on it and
> > you
> > > > > > explain the actual implementation to introduce this change.
> > > > > > Still I do not fully understand why it was decided to work on top
> > of
> > > > XML
> > > > > by
> > > > > > filtering/enhancing it instead of working at the POM (in
> > > > > > memory datamodel) level.
> > > > > > With the current understanding I have, by doing this choice of
> > > working
> > > > at
> > > > > > XML level, it looks like it was decided to bypass (if not kill)
> > core
> > > > > > extensions that enhance the POM itself and not the pom.xml ;
> here I
> > > can
> > > > > > think of (but probably not limited to):
> > > > > > - polyglot-maven: do not use XML but other format to describe the
> > POM
> > > > > > (yaml, json, kotlin, java, other XML formats, ...)
> > > > > > - jgitver-maven-plugin (or forks like
> > > maven-git-versioning-extension):
> > > > > > dynamic computation of projects version based on git history
> > > > > >
> > > > > > With the introduction of core extensions, I thought it was a move
> > to
> > > > open
> > > > > > the internals and let externals contribute to the capabilities of
> > > > maven.
> > > > > > With the move to a XML handling chain, I see it as a
> > > > restriction/regress
> > > > > in
> > > > > > favor of core closed functionalities. An example of that is what
> is
> > > > > > provided as CIFriendly stuff, IMO it could/should have been
> > provided
> > > > by a
> > > > > > plugin/extension but instead it is hard written in maven core and
> > is
> > > > not
> > > > > > opened for external contribution (plugin/extension I mean).
> > > > > >
> > > > > > Perhaps I am totally wrong but I think that maven core should
> > define
> > > > all
> > > > > > its expectations at an API level so that extensions/plugins could
> > > hook
> > > > at
> > > > > > this API level. The default packaging of maven could/should
> provide
> > > > > default
> > > > > > implementations of those expectations (for example reading a
> > pom.xml
> > > > file
> > > > > > to a POM model, dumping a POM to a pom-4.0.0.xml,
> > > > transforming/reducing a
> > > > > > POM to POM-consumer, dumping POM-consumer to
> > pom-consumer-5.0.0.xml,
> > > > ...)
> > > > > > and let extensions/plugins/default implementations work along the
> > > build
> > > > > > process with the API & POMs to provide different features and
> > > > > capabilities.
> > > > > >
> > > > > > Matthieu
> > > > > >
> > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote:
> > > > > >
> > > > > > > I've made a recording[1] about it, which hopefully answers most
> > > > > > questions.
> > > > > > >
> > > > > > > Robert
> > > > > > >
> > > > > > > [1] https://youtu.be/KDAmlNKZJto
> > > > > > >
> > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard
> > > > > > > wrote:
> > > > > > > > Not exactly sure what work you mean
> > > > > > > everything related to maven-xml: Build/ConsumerPomXMLFilterxxx,
> > > > > > > Build/ConsumerModelSourcexxxx and the transformer stuff.
> > > > > > > Especially, when looking at classes like CiFriendlyXMLFilter, I
> > > would
> > > > > > have
> > > > > > > thought that such things could have been done elsewhere,
> working
> > on
> > > > the
> > > > > > > object model (not on the XML stuff) especially for the BuildPom
> > > part.
> > > > > > >
> > > > > > > > however specifically the consumer POM integrates with so many
> > > > > external
> > > > > > > ecosystems
> > > > > > > We're aligned here, this has to be stable and well defined by a
> > > > schema.
> > > > > > >
> > > > > > > Matthieu
> > > > > > >
> > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hello,
> > > > > > > >
> > > > > > > > Not exactly sure what work you mean and I fully agree that
> > using
> > > a
> > > > > core
> > > > > > > > model should still be the API for plugins and extensions to
> > work
> > > > > with,
> > > > > > > > however specifically the consumer POM integrates with so many
> > > > > external
> > > > > > > > ecosystems, I would expect it to be defined in terms of XML
> > > Schema
> > > > > with
> > > > > > > > explicite semantic (and the inherent compatibility with
> exiting
> > > > > POMs).
> > > > > > > >
> > > > > > > > Gruss
> > > > > > > > Bernd
> > > > > > > > --
> > > > > > > > http://bernd.eckenfels.net
> > > > > > > > ________________________________
> > > > > > > > Von: Matthieu BROUILLARD
> > > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM
> > > > > > > > An: dev@maven.apache.org
> > > > > > > > Betreff: maven 4.0.0 new XML stuff
> > > > > > > >
> > > > > > > > Hello all,
> > > > > > > >
> > > > > > > > regarding the active work occurring for maven 4.0.0 I noticed
> > the
> > > > > > > > introduction of a lot of new stuff around SAX parsing &
> > > filtering.
> > > > > > > > I am wondering if that means that it was decided that the
> input
> > > > > format
> > > > > > of
> > > > > > > > maven projects will be XML forever meaning probably, among
> > > others,
> > > > > the
> > > > > > > end
> > > > > > > > of polyglot extensions.
> > > > > > > > Could you explain such a move (or point to
> rationals/documents)
> > > and
> > > > > why
> > > > > > > you
> > > > > > > > did not leverage working on the in memory object model
> allowing
> > > > > > > > extensions/plugins to contribute/hook in the chain of
> building
> > > the
> > > > > > > BuildPOM
> > > > > > > > & ConsumePOM? In the past I really thought that this move to
> > > 'Build
> > > > > vs
> > > > > > > > Consumer' POM would make clear separations between the input
> > > format
> > > > > of
> > > > > > > > descriptors and the core system but I perhaps misunderstood
> > > things.
> > > > > > > >
> > > > > > > > Also, are there plans regarding the future of core
> extensions?
> > > > > > > > With core extensions it was possible to hook into the POM
> model
> > > > > loading
> > > > > > > and
> > > > > > > > do transformations to do dynamic changes but by working on
> the
> > > XML
> > > > > > > directly
> > > > > > > > I see a shift (if not red stop) in this
> contribution/delegation
> > > > > > > mechanism.
> > > > > > > >
> > > > > > > > Thanks for your time & answers.
> > > > > > > >
> > > > > > > > Matthieu Brouillard
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
