https://github.com/apache/maven-integration-testing/tree/master/core-it-suite/src/test/resources/mng-6957-buildconsumer is the most complete IT
On 4-1-2021 12:59:51, Romain Manni-Bucau <rmannibu...@gmail.com> wrote: Le lun. 4 janv. 2021 à 12:36, Robert Scholte a écrit : > There's just one thing I want to say: > I'm having trouble with the term "broken". > Well, literally meant broken as decorelated from the user intent and extension model. Anyway, didn't intend to blame but more identify the blockers for a GA so point was really that it seem that on the two sides only the producing one is not yet ready since it keeps does not sanitize the model completely and keeps build only data like comments, right? Also not yet clear for me if we loose the extension enrichments there. > If a Maven project could be built with M3.6.3, it can still be built with > M4. > If not, it is either regression (MNG-6957, MNG-7063) which must be fixed, > or it requires changes to a plugin for understandable reasons > (maven-pgp-plugin) > AFAIK an interesting extension like the maven-tiles has been tested and > still works. > Do you have this handy, is it in our test suite? I'd like to check the produced pom matches the enriched model but happy to start from something already there. > > Robert > > On 3-1-2021 19:35:25, Romain Manni-Bucau wrote: > Le dim. 3 janv. 2021 à 19:04, Robert Scholte a > écrit : > > > I don't remember all those details anymore, because I hit those in the > > beginning. > > Trying things over and over again I decided that this is probably the > most > > successful approach. > > > > > > What of the goals was to keep the pom.xml as is as much as possible. > > We can only decide for the specific Maven elements how to handle them, we > > should not decide about comments and licenses. > > BTW, the license issue was hard to solve. You cannot use it from the > pom's > > , because there might be multiple licenses. > > > > I disagree, it is saner IMO to evolve to support that than doing > anything else. > Once again you keep things which don't make sense in a consumed pom in > current impl so i'd say the sucess in a few cases breaks as much cases so > we need to revisit anyway IMHO. > > > > > > The current implementation is a solid way to ensure we're not breaking > too > > much, because Maven controls the XML filters. > > > > Hmm, breaking extensions seems to break too much (I'm not speaking of other > parts which breaks the ecosystem there but just that is sufficient IMHO to > say we must check back our solution). > > > > > > Also keep in mind, that I only want Maven to decide which modifications > > are done. > > > > For the consumed pom I agree but it is consistent with keeping everything > working instead of breaking too, no? > > > > > > Current polyglot projects should still work, but they cannot benefit from > > the build/consumer functions yet. > > > > So pom -> build model is kept, build model -> produced pom is broken? Is it > the complete status? > Sounds ok for a 4.0 and a 4.1 can fix it if so. > Just want to ensure first part is not broken at all. > > > > > > Robert > > On 3-1-2021 16:38:38, Romain Manni-Bucau wrote: > > Le dim. 3 janv. 2021 à 16:18, Robert Scholte a > > écrit : > > > > > > So what I was expecting was: raw xml model -> converted to unified > > > consumed model -> extensions -> model processing. > > > > > > This is only the build pom part. You're missing the consume part, where > > > the xml is distributed. > > > > > > > Yes but with previous chain the consume part is "clean/normalize -> dump" > > since we are using consumed model - only standard model - in memory > > already. > > > > > > > Build is raw + enrich, consumer is raw + enrich + reduce (removing > > > relativePath and modules are the first examples, but much more is > > possible) > > > > > > Going for the in memory was also my first thought, but I would loose > > > information, hence I came up with the current implementation. > > > > > > > I don't see what you loose ot be honest. > > You mentionned license but this one is in the pom so not a big deal, > > comments which are undesired IMHO as mentionned and element order which > can > > really be discussed since we can desire to enforce an order to normalize > > consumption + it shouldn't be important since from the project point of > > view your pom is already "broken"/lost (as all your intelligence is lost > by > > this "not passthrough" process). > > So overall I don't see what you would loose from the consumer side but I > > see what you lost from maven ecosystem side. > > > > > > > Again, we're at a point where we can have counter solutions, but don't > > > expect me to implement it. > > > > > > > For now I'm just trying to ensure we agree we don't want to break > existing > > extensions and the nice ecosystem we built after years. > > This was really a move forward and it sounds like we broke it at maven 4 > > without any user gain which sounds terrible. > > > > > > > > > > thanks, > > > Robert > > > On 3-1-2021 15:25:21, Romain Manni-Bucau wrote: > > > Hi all, > > > > > > I kind of join Matthieu thoughts there, there is no point to work at > xml > > > level to create the consumed pom - comments is not a point since it can > > > commonly/easily refer to a dropped part of the pom so they should be > > > stripped. > > > Current extension model got proven adapted and adopted, using a lower > > level > > > extension API will not since XML is, even if still mainstream, often > > > replaced by alternative configurations and to have done the work to > > inject > > > XML configuration programmatically compred to current option, it is a > > pain. > > > The in memory model should stick to consumed model IMHO - being > > > programmatic there is no point to make it easier, worse case we can add > > > helper beans (injectable) but in terms of model it will not help. > > > > > > So what I was expecting was: > > > > > > raw xml model -> converted to unified consumed model -> extensions -> > > model > > > processing. > > > > > > Indeed, real chain adds a small processing over the first arrow (inject > > > versions for example) but nothing crazy and breaking this overall flow > > > which stays user friendly. > > > > > > Strictly speaking the new model is just a built-in extension for me > which > > > is particular because it will enforce IDE to integrate a new format - > > > wheres polyglot extensions or others don't require static analyzis by > > > themself not being "standard". > > > > > > That said, there is nothing crazy with current implementation, it just > > > require to be updated to be able to take extension changes into > account. > > > This can be done by making the extension model 'spyable' (ie if a > > > dependency/plugin is added it will be reflected in the final written > > > pom.xml). > > > This sounds - instrumenting the extension model API or doing a diff > after > > > extension phase - like a compromise and let people gets the best of > both > > > worlds to me. > > > > > > Wdyt? > > > > > > Romain Manni-Bucau > > > @rmannibucau | Blog > > > | Old Blog > > > | Github | > > > LinkedIn | Book > > > > > > > > > > > > Le dim. 3 janv. 2021 à 14:46, Robert Scholte a > > > écrit : > > > > > > > Hi Matthieu, > > > > > > > > As you understand, something had to be changed to move Maven forward. > > > > I've decided to pick up that challenge and came up with the current > > > > solution. > > > > > > > > My main concerns was that I wanted to keep the fileModel as much as > is. > > > > That includes the license, comments and element order. > > > > This information if not available in the memory model, so I needed > the > > > > original pom file. > > > > With that in mind, the usage of XMLFilters looks like the most > > > appropriate > > > > solution. > > > > > > > > I am certain that XML is still the most used format, so if we can > have > > > > improvements for those users, I'm already very happy. > > > > > > > > And yes, there are plugins that needs to be updated, but doing > nothing > > is > > > > not an option anymore. > > > > > > > > There are more people that share their concerns, but it took me > several > > > > years to reach this point. > > > > We now have something that seems to work, anybody who can improve or > > can > > > > come up with an alternative implementation can do so. > > > > > > > > thanks, > > > > Robert > > > > On 3-1-2021 12:55:41, Matthieu Brouillard wrote: > > > > Thanks Robert for the video link. > > > > > > > > I fully understand the rationales behind the separation of > > > > build/consumer pom and the video provides some insights on it and you > > > > explain the actual implementation to introduce this change. > > > > Still I do not fully understand why it was decided to work on top of > > XML > > > by > > > > filtering/enhancing it instead of working at the POM (in > > > > memory datamodel) level. > > > > With the current understanding I have, by doing this choice of > working > > at > > > > XML level, it looks like it was decided to bypass (if not kill) core > > > > extensions that enhance the POM itself and not the pom.xml ; here I > can > > > > think of (but probably not limited to): > > > > - polyglot-maven: do not use XML but other format to describe the POM > > > > (yaml, json, kotlin, java, other XML formats, ...) > > > > - jgitver-maven-plugin (or forks like > maven-git-versioning-extension): > > > > dynamic computation of projects version based on git history > > > > > > > > With the introduction of core extensions, I thought it was a move to > > open > > > > the internals and let externals contribute to the capabilities of > > maven. > > > > With the move to a XML handling chain, I see it as a > > restriction/regress > > > in > > > > favor of core closed functionalities. An example of that is what is > > > > provided as CIFriendly stuff, IMO it could/should have been provided > > by a > > > > plugin/extension but instead it is hard written in maven core and is > > not > > > > opened for external contribution (plugin/extension I mean). > > > > > > > > Perhaps I am totally wrong but I think that maven core should define > > all > > > > its expectations at an API level so that extensions/plugins could > hook > > at > > > > this API level. The default packaging of maven could/should provide > > > default > > > > implementations of those expectations (for example reading a pom.xml > > file > > > > to a POM model, dumping a POM to a pom-4.0.0.xml, > > transforming/reducing a > > > > POM to POM-consumer, dumping POM-consumer to pom-consumer-5.0.0.xml, > > ...) > > > > and let extensions/plugins/default implementations work along the > build > > > > process with the API & POMs to provide different features and > > > capabilities. > > > > > > > > Matthieu > > > > > > > > On Thu, Dec 31, 2020 at 7:01 PM Robert Scholte wrote: > > > > > > > > > I've made a recording[1] about it, which hopefully answers most > > > > questions. > > > > > > > > > > Robert > > > > > > > > > > [1] https://youtu.be/KDAmlNKZJto > > > > > > > > > > On 31-12-2020 16:18:57, Matthieu Brouillard > > > > > wrote: > > > > > > Not exactly sure what work you mean > > > > > everything related to maven-xml: Build/ConsumerPomXMLFilterxxx, > > > > > Build/ConsumerModelSourcexxxx and the transformer stuff. > > > > > Especially, when looking at classes like CiFriendlyXMLFilter, I > would > > > > have > > > > > thought that such things could have been done elsewhere, working on > > the > > > > > object model (not on the XML stuff) especially for the BuildPom > part. > > > > > > > > > > > however specifically the consumer POM integrates with so many > > > external > > > > > ecosystems > > > > > We're aligned here, this has to be stable and well defined by a > > schema. > > > > > > > > > > Matthieu > > > > > > > > > > On Thu, Dec 31, 2020 at 3:59 PM Bernd Eckenfels > > > > > wrote: > > > > > > > > > > > Hello, > > > > > > > > > > > > Not exactly sure what work you mean and I fully agree that using > a > > > core > > > > > > model should still be the API for plugins and extensions to work > > > with, > > > > > > however specifically the consumer POM integrates with so many > > > external > > > > > > ecosystems, I would expect it to be defined in terms of XML > Schema > > > with > > > > > > explicite semantic (and the inherent compatibility with exiting > > > POMs). > > > > > > > > > > > > Gruss > > > > > > Bernd > > > > > > -- > > > > > > http://bernd.eckenfels.net > > > > > > ________________________________ > > > > > > Von: Matthieu BROUILLARD > > > > > > Gesendet: Thursday, December 31, 2020 3:19:09 PM > > > > > > An: dev@maven.apache.org > > > > > > Betreff: maven 4.0.0 new XML stuff > > > > > > > > > > > > Hello all, > > > > > > > > > > > > regarding the active work occurring for maven 4.0.0 I noticed the > > > > > > introduction of a lot of new stuff around SAX parsing & > filtering. > > > > > > I am wondering if that means that it was decided that the input > > > format > > > > of > > > > > > maven projects will be XML forever meaning probably, among > others, > > > the > > > > > end > > > > > > of polyglot extensions. > > > > > > Could you explain such a move (or point to rationals/documents) > and > > > why > > > > > you > > > > > > did not leverage working on the in memory object model allowing > > > > > > extensions/plugins to contribute/hook in the chain of building > the > > > > > BuildPOM > > > > > > & ConsumePOM? In the past I really thought that this move to > 'Build > > > vs > > > > > > Consumer' POM would make clear separations between the input > format > > > of > > > > > > descriptors and the core system but I perhaps misunderstood > things. > > > > > > > > > > > > Also, are there plans regarding the future of core extensions? > > > > > > With core extensions it was possible to hook into the POM model > > > loading > > > > > and > > > > > > do transformations to do dynamic changes but by working on the > XML > > > > > directly > > > > > > I see a shift (if not red stop) in this contribution/delegation > > > > > mechanism. > > > > > > > > > > > > Thanks for your time & answers. > > > > > > > > > > > > Matthieu Brouillard > > > > > > > > > > > > > > > > > > > > >
