...and one more thing: Sorry for bombing this thread, and I see we have clear counter-points for each other, so here is a proposal: long time ago there was an institution of "Maven Hangouts", where regular (or irregular) face2face meetings were held between not only Maven devs, but any interested parties (to hear about latest Maven developments). So, let's revive this custom, and for the first session let me prepare a demo of my idea, show it, and Romain and others can see what I mean, but also can ask or show counter examples and so on. If something crops out that needs more time, we can continue on to the next session, and so on... Of course, if anyone is willing to show how "jpms dev is done today" is welcome as well!
History shows that having lengthy ML discussions usually dies off into silence. I think we could use Google Meet, where the 60min limit is actually even welcome, to not get into marathon meetings and stay focused (or continue on the next session). WDYT? On Sun, Oct 29, 2023 at 4:31 PM Tamás Cservenák <ta...@cservenak.net> wrote: > === > > Digression: I _think_ we have a misconception present, so excuse me for > this digression, but I have to write it down here, just to make things > clear. > Is basically represented by this issue > https://issues.apache.org/jira/browse/MNG-7373 > > tl;dr: It is common misconception, that Maven works like this: > - in POM you declare dependencies > - Maven, when invoked, resolves them by getting the POM > - with POM, it reads the packaging out of it, and "decides" what to do > more (ie. get the JAR) > > This is NOT how Maven works nor has ever worked like it. > > For Maven what matters is how you DECLARE your dependency, that is the > instruction it follows. The remote POM may even have packaging=kangaroo, it > will NOT bother with it. All that matters is how you declare the kangaroo > packaged artifact, if you do it as type=jar, it will go for JAR. > > (Reason is trivial: you can produce JAR artifacts in a million ways, using > packaging "jar" that is built into Maven core, but also using "takari-jar", > or "bundle", or some in-house packaging "super-jar" or "my-super-duper-jar" > or whatever. Naturally, to consume these types, Maven would need to know > about these types, hence, it would need extensions for it to be installed. > In short: to make Maven work like in common misconception imagined, it > would need to have ALL the extensions for ALL the packaging types out > there, that produce files with extension "jar"). > > In short packaging != type. > > On Sun, Oct 29, 2023 at 4:10 PM Tamás Cservenák <ta...@cservenak.net> > wrote: > >> Howdy, >> >> The current draft of types we want to introduce (and packaging): >> https://gist.github.com/cstamas/4e9bcbef25ce912a90ad1e127b0c5db8 >> >> === >> >> Romain, I don't understand your "This is wrong, downstream is either >> module or jar", as it was actually you and your example that mentioned >> "once put it here, once put it there". Nothing is lost IMHO, just like in >> case of "takari-jar" nothing is lost. >> >> Or if we misunderstood each other: by "downstream" I mean "down the road, >> when a project being built, is about to be consumed as dependency". >> >> And the point is, that exactly due ArtifactHandler/ArtifactType/Type (in >> mvn4), Maven "sees" it as "jar" or "module" or whatever, but for resolver, >> those two are _same thing_. It is _same file_. And this is the crux, as for >> the resolver, it is really about getting that one file, while the type (for >> Maven) tells HOW to make use of it. So for resolving, there is no any kind >> of "lost" information, again, the very same way it works for "takari-jar". >> >> T >> >> On Sun, Oct 29, 2023 at 3:52 PM Romain Manni-Bucau <rmannibu...@gmail.com> >> wrote: >> >>> Le dim. 29 oct. 2023 à 15:29, Tamás Cservenák <ta...@cservenak.net> a >>> écrit : >>> >>> > Howdy, >>> > >>> > So to return to the "root" idea (of Maven), Maven is "declarative", >>> where >>> > users should declare what they want, it should most certainly not >>> "guess" >>> > what user intent is. As long as we have "magical implicit guesswork" >>> (like >>> > that in javadoc) present in process, it is bad, as that means we do not >>> > allow our users to express their goal. >>> > >>> >>> Yes and i liked that but we broke it with forcing plugin version locking >>> (for good) so we can need to revise our root too to match current world >>> which is no more unique, makes years we ignore that fact but it already >>> blows up. >>> >>> So my 2cts are we cant by design here, we tried hard and failed, not >>> technically but by design. >>> >>> >>> >>> > Originally Mojos were envisioned to be simple, focused, doing one >>> thing and >>> > doing it well (a la UNIX tools). Some plugins went in the total >>> opposite >>> > direction, as they became Godzilla plugins (with unmaintainable >>> complex and >>> > large amounts of "logic" -- guess logic and bloated codebase) >>> targeting to >>> > solve "everything". This also resulted in our users assuming "every >>> problem >>> > should have a corresponding Mojo" (this also steered toward bloated, >>> over >>> > complex Mojos), where many many other aspects and capabilities of Maven >>> > were totally neglected, like lifecycle, custom packaging and so on. >>> > >>> > In short, Romain, yes, today, you CAN build all sort of things in >>> "smart >>> > way", just like cooking a soup: get a good base (packaging), add a >>> little >>> > bit of this (mojo A) and a little bit of that (mojo B) and voila, you >>> will >>> > end up with a "soup". And it works, yes, but this "smart" way has many >>> > pitfalls along the way, with most problematic of not being explicit. >>> But by >>> > doing that, your build becomes Ant-ish (imperative-ish), and you are >>> > sliding off the declarative path. >>> > >>> >>> For cases needing it and all the mentionned ones are (and will) NOT (be) >>> mainstream. >>> So all good IMHO. >>> >>> >>> > Also, you ARE aware that if you build a project w/ packaging=module >>> (that >>> > as output has an artifact with extension jar), you DON'T HAVE TO >>> address it >>> > downstream (when you depend on it) as "module", right? This is the >>> actual >>> > reason why I brought up Takari Lifecycle, as there you are building >>> > projects with packaging=takari-jar, but when you consume those, you >>> refer >>> > to them as type=jar, and not as type=takari-jar, right? >>> > >>> >>> This is wrong, downstream is either module or jar, here you loose if it >>> is >>> a module transitively and module assume all transitive deps are which is >>> very often wrong so it does not work downstream. >>> >>> >>> > So, I have a feeling that you bring bold conclusions (like "Your >>> solution >>> > is valid technically but does not solve the original issue"), you do >>> not >>> > really understand what I am trying to say here. >>> > >>> >>> Hooe you are true but trust me i went where you are and my conclusion is >>> it >>> cant be done right for enough projects to be worth it. >>> >>> >>> > Also, "this" certainly does NOT "work well today" (even in this thread >>> poor >>> > tooling mentioned), but true, you can hack-around it, or alike, but >>> none of >>> > current solutions are explicit, declarative and naturally expressed in >>> > Maven (but are bolted on). >>> > >>> > Finally, having anything "on top" of resolver is not gonna help >>> anything, >>> > as Resolver is really about Artifacts only (Everything is an >>> Artifact!) and >>> > it is Maven on top of Resolver that should interpret these artifacts >>> based >>> > on user posed instructions (declarations). Resolver is really just >>> about >>> > "getting" and "putting" artifacts, not USING them. >>> > >>> >>> It foes cause it is not aboug resolver but enabling user to express path >>> with maven abstraction, not resolution itself. >>> >>> Java is about paths, maven coordinates, we miss a link on user land to >>> make >>> it smooth. >>> >>> >>> > Thanks >>> > T >>> > >>> > >>> > On Sun, Oct 29, 2023 at 2:56 PM Tamás Cservenák <ta...@cservenak.net> >>> > wrote: >>> > >>> > > Romain, >>> > > >>> > > it's probably me, but I have no faintest idea what you are trying to >>> > say... >>> > > >>> > > What do you mean by "standalone"? >>> > > What is the wrong packaging? >>> > > Why will I lose the ability to specify where it goes? Also, as I said >>> > > before, if you list project/deps gav:jar AND gav:module, you would be >>> > > putting _one same JAR_ on both paths (would you really want that?) >>> > > Also, here we are speaking about _dependencies_ but you suddenly >>> switch >>> > to >>> > > building a project? >>> > > >>> > > But one thing for sure: we need _less_ "guess logic" and not _more of >>> > it_. >>> > > >>> > > I think we are not talking about the same thing(s) here, but again, >>> it's >>> > > just maybe me. >>> > > >>> > > T >>> > > >>> > > On Sun, Oct 29, 2023 at 1:32 PM Romain Manni-Bucau < >>> > rmannibu...@gmail.com> >>> > > wrote: >>> > > >>> > >> Le dim. 29 oct. 2023 à 12:44, Tamás Cservenák <ta...@cservenak.net> >>> a >>> > >> écrit : >>> > >> >>> > >> > Can you provide a real example? As I don't quite understand, so >>> you >>> > >> would >>> > >> > have a dependency (a fat spring boot jar), that is a "module dep >>> is a >>> > >> > module in compile/some tests but not at runtime (spring boot >>> fatjar)". >>> > >> So >>> > >> > all this within one maven module (compile/test/runtime?). >>> > >> > >>> > >> >>> > >> >>> > >> You have an app (src/main/java), all the chain down the line uses >>> > modules >>> > >> cause compiler is standalone, surefire (junit-launcher) is >>> standalone, >>> > >> javadoc etc...and the packaging phase (jar/war/fatjar/...) is not >>> > >> standalone. >>> > >> Then using module will make your build pass but your IT (if you have >>> > else >>> > >> your runtime) will not use that kind of construction/runtime. >>> > >> the issue is as soon as you mark them modules then it must be >>> module for >>> > >> all plugins and therefore you will get a wrong packaging (think bnd >>> for >>> > >> ex) >>> > >> or said otherwise you loose the consumption ability the JVM has >>> > providing >>> > >> both classpath and module path for the same jar (jlink requires >>> deps to >>> > be >>> > >> modules but these modules can be used in the classpath most of the >>> time, >>> > >> in >>> > >> particular in tests where it helps in several scenarii like >>> mocking). >>> > >> >>> > >> These are all valid features we don't want to break in maven. >>> > >> >>> > >> The consuming side is problematic since you restart from scratch, >>> all >>> > the >>> > >> jpms meta are to throw away cause we don't want to break the model >>> on >>> > one >>> > >> side and on another side it depends the consumer the way you >>> consume it >>> > >> and >>> > >> dispatch the dependency on the classpath or module path. >>> > >> >>> > >> Your solution is valid technically but does not solve the original >>> > issue, >>> > >> it just moves it elsewhere IMHO. >>> > >> >>> > >> This already works well today at the cost of being explicit in the >>> > plugins >>> > >> configs and with your proposal it will still work at the same cost >>> > (maybe >>> > >> reversed). >>> > >> >>> > >> So ultimately I don't think it is a dep meta we need but more a >>> wrapper >>> > on >>> > >> top of the resolver (the guess logic we have in plexus for ex) which >>> > >> should >>> > >> be easier to configure, maybe globally for the project and >>> ultimately >>> > per >>> > >> plugin (think "configuration" in gradle world of maven depencies >>> set in >>> > >> our >>> > >> world)....or we just don't do anything and ease the dependency >>> > referencing >>> > >> (gav->path) to ease this explicit configuration - a bit like >>> including >>> > >> dependencies:properties in core by default. >>> > >> >>> > >> >>> > >> >>> > >> >>> > >> > >>> > >> > T >>> > >> > >>> > >> > On Sun, Oct 29, 2023 at 12:37 PM Romain Manni-Bucau < >>> > >> rmannibu...@gmail.com >>> > >> > > >>> > >> > wrote: >>> > >> > >>> > >> > > Le dim. 29 oct. 2023 à 12:12, Tamás Cservenák < >>> ta...@cservenak.net> >>> > a >>> > >> > > écrit : >>> > >> > > >>> > >> > > > Given that jar (spring boot fatjar) is once this once that, >>> you >>> > >> refer >>> > >> > to >>> > >> > > it >>> > >> > > > in deps as needed: >>> > >> > > > in one module is fat:jar in other is fat:module, as needed. >>> > >> > > > >>> > >> > > > You are the one explicitly telling what you want. >>> > >> > > > >>> > >> > > >>> > >> > > In my example (and spring boot is not the only one, wars are >>> another >>> > >> > common >>> > >> > > one, or any java command more generally) the two modules are a >>> > single >>> > >> > > one....and no doing two module would be worse than what we have >>> > today. >>> > >> > > >>> > >> > > >>> > >> > > > Thanks >>> > >> > > > T >>> > >> > > > >>> > >> > > > On Sun, Oct 29, 2023 at 8:42 AM Romain Manni-Bucau < >>> > >> > > rmannibu...@gmail.com> >>> > >> > > > wrote: >>> > >> > > > >>> > >> > > > > Interesting but common case: a module dep is a module in >>> > >> compile/some >>> > >> > > > tests >>> > >> > > > > but not at runtime (spring boot fatjar). >>> > >> > > > > Back to explicit config in plugins and drop new module type? >>> > >> > > > > >>> > >> > > > > Le dim. 29 oct. 2023 à 07:46, Christoph Läubrich < >>> > >> > m...@laeubi-soft.de> >>> > >> > > a >>> > >> > > > > écrit : >>> > >> > > > > >>> > >> > > > > > > where properties are totally extensible, >>> > >> > > > > > >>> > >> > > > > > And if now I could supply additional properties from the >>> > >> xml-model >>> > >> > > ... >>> > >> > > > > > >>> > >> > > > > > Am 29.10.23 um 00:40 schrieb Tamás Cservenák: >>> > >> > > > > > > And finally this is hardly gonna happen in Maven 3 >>> lifespan, >>> > >> as >>> > >> > > sadly >>> > >> > > > > > > ArtifactHandler of it is quite limited: has only one >>> flag: >>> > >> > > > > > > >>> > >> > > > > > >>> > >> > > > > >>> > >> > > > >>> > >> > > >>> > >> > >>> > >> >>> > >>> https://github.com/apache/maven/blob/maven-3.9.x/maven-artifact/src/main/java/org/apache/maven/artifact/handler/ArtifactHandler.java#L55 >>> > >> > > > > > > >>> > >> > > > > > > Sadly, Maven4 Type continued on this path, but luckily >>> we >>> > are >>> > >> in >>> > >> > > > alpha, >>> > >> > > > > > and >>> > >> > > > > > > will propose a PR to type that currently looks like >>> this: >>> > >> > > > > > > >>> > >> > > > > > >>> > >> > > > > >>> > >> > > > >>> > >> > > >>> > >> > >>> > >> >>> > >>> https://github.com/apache/maven/blob/master/api/maven-api-core/src/main/java/org/apache/maven/api/Type.java#L80 >>> > >> > > > > > > >>> > >> > > > > > > And would rework it to be more (if not same as) the >>> resolver >>> > >> > > > > > ArtifactType: >>> > >> > > > > > > >>> > >> > > > > > >>> > >> > > > > >>> > >> > > > >>> > >> > > >>> > >> > >>> > >> >>> > >>> https://github.com/apache/maven-resolver/blob/master/maven-resolver-api/src/main/java/org/eclipse/aether/artifact/ArtifactType.java#L63 >>> > >> > > > > > > >>> > >> > > > > > > where properties are totally extensible, for example >>> "add to >>> > >> > > > classpath" >>> > >> > > > > > is >>> > >> > > > > > > really just one flag (added by ArifactType): >>> > >> > > > > > > >>> > >> > > > > > >>> > >> > > > > >>> > >> > > > >>> > >> > > >>> > >> > >>> > >> >>> > >>> https://github.com/apache/maven-resolver/blob/master/maven-resolver-api/src/main/java/org/eclipse/aether/artifact/ArtifactProperties.java#L58 >>> > >> > > > > > > >>> > >> > > > > > > So, ArtifactProperties could be extended with >>> > >> > > > "constitutesModulePath", >>> > >> > > > > > > "agent" and so on... To make this really extensible. >>> > >> > > > > > > >>> > >> > > > > > > In maven3 the ArtifactHandler this makes it impossible. >>> > There >>> > >> is >>> > >> > > > still >>> > >> > > > > > hope >>> > >> > > > > > > in Maven 4 >>> > >> > > > > > > >>> > >> > > > > > > Thanks >>> > >> > > > > > > T >>> > >> > > > > > > >>> > >> > > > > > > >>> > >> > > > > > > On Sun, Oct 29, 2023 at 12:32 AM Tamás Cservenák < >>> > >> > > > ta...@cservenak.net> >>> > >> > > > > > > wrote: >>> > >> > > > > > > >>> > >> > > > > > >> This would also mean, that if you have a dependency >>> that is >>> > >> > > already >>> > >> > > > > JPMS >>> > >> > > > > > >> modularized (is java9+ and has module-info), then: >>> > >> > > > > > >> >>> > >> > > > > > >> a) if you declare it as type="jar" it means you want >>> to put >>> > >> it >>> > >> > on >>> > >> > > > > > >> classpath (use it as "plain old jar") >>> > >> > > > > > >> b) if you declare it as type="module" it means you >>> want it >>> > on >>> > >> > > > > modulepath >>> > >> > > > > > >> etc >>> > >> > > > > > >> >>> > >> > > > > > >> On Sun, Oct 29, 2023 at 12:30 AM Tamás Cservenák < >>> > >> > > > ta...@cservenak.net >>> > >> > > > > > >>> > >> > > > > > >> wrote: >>> > >> > > > > > >> >>> > >> > > > > > >>> Of course, the logic HOW and WHAT to make with these >>> would >>> > >> be >>> > >> > > > needed >>> > >> > > > > to >>> > >> > > > > > >>> be added to javadoc, compiler and all the plugins that >>> > need >>> > >> to >>> > >> > > > > > distinguish. >>> > >> > > > > > >>> But this would stop any need for any heuristic, >>> guesswork, >>> > >> > > > > smart-ness, >>> > >> > > > > > >>> etc... >>> > >> > > > > > >>> >>> > >> > > > > > >>> OTOH, if we introduce new packaging lifecycle "module" >>> > (so a >>> > >> > > > project >>> > >> > > > > > that >>> > >> > > > > > >>> builds module would do project/packaging=module), it >>> could >>> > >> > nicely >>> > >> > > > > > enforce >>> > >> > > > > > >>> things like: >>> > >> > > > > > >>> - prevent non allowed packages >>> > >> > > > > > >>> - enforce presence of module-info.class (maybe some >>> light >>> > >> > > > > verification) >>> > >> > > > > > >>> - ensure project is Java9+ etc >>> > >> > > > > > >>> >>> > >> > > > > > >>> Most of this was somewhat done in Takari Lifecycle >>> (also >>> > >> with >>> > >> > > > custom >>> > >> > > > > > >>> packaging like "takari-jar" was). >>> > >> > > > > > >>> >>> > >> > > > > > >>> >>> > >> > > > > > >>> T >>> > >> > > > > > >>> >>> > >> > > > > > >>> On Sun, Oct 29, 2023 at 12:26 AM Tamás Cservenák < >>> > >> > > > > ta...@cservenak.net> >>> > >> > > > > > >>> wrote: >>> > >> > > > > > >>> >>> > >> > > > > > >>>> So, basically this is what am proposing: >>> > >> > > > > > >>>> >>> > >> > > >>> https://gist.github.com/cstamas/76f262538b5a11f6ee23d6d8c86f10ec >>> > >> > > > > > >>>> >>> > >> > > > > > >>>> Basically, Maven core (and hence plugins) could >>> > distinguish >>> > >> > > among >>> > >> > > > > > >>>> different "types" of dependencies (while would all >>> still >>> > be >>> > >> > > plain >>> > >> > > > > > JARs). >>> > >> > > > > > >>>> So "jar" would be put on classpath, "module" on >>> module >>> > >> path, >>> > >> > > > "agent" >>> > >> > > > > > >>>> would got special treatment and so on. >>> > >> > > > > > >>>> >>> > >> > > > > > >>>> Point is to _differentiate_. >>> > >> > > > > > >>>> >>> > >> > > > > > >>>> T >>> > >> > > > > > >>>> >>> > >> > > > > > >>>> On Sun, Oct 29, 2023 at 12:21 AM Tamás Cservenák < >>> > >> > > > > ta...@cservenak.net >>> > >> > > > > > > >>> > >> > > > > > >>>> wrote: >>> > >> > > > > > >>>> >>> > >> > > > > > >>>>> Unsure from where you get that, but is wrong >>> conclusion. >>> > >> > > > > > >>>>> >>> > >> > > > > > >>>>> You can have dep1:jar, dep2:module, dep3:agent and >>> all 3 >>> > >> MAY >>> > >> > > > > > >>>>> (ArtifactHandler dependent, assuming "jar", >>> "module" and >>> > >> > > "agent" >>> > >> > > > > > artifact >>> > >> > > > > > >>>>> handlers all return extension=jar) refer to the >>> same JAR >>> > >> file >>> > >> > > in >>> > >> > > > > > your local >>> > >> > > > > > >>>>> repository. >>> > >> > > > > > >>>>> >>> > >> > > > > > >>>>> Type merely adds "semantics" WHAT is it about, HOW >>> to >>> > make >>> > >> > use >>> > >> > > of >>> > >> > > > > it. >>> > >> > > > > > >>>>> Please see >>> > >> > > > > > >>>>> >>> > >> > > > > > >>> > >> > > > > >>> > >> > > > >>> > >> > > >>> > >> > >>> > >> >>> > >>> https://maven.apache.org/repositories/artifacts.html#but-where-do-i-set-artifact-extension >>> > >> > > > > > >>>>> >>> > >> > > > > > >>>>> Thanks >>> > >> > > > > > >>>>> T >>> > >> > > > > > >>>>> >>> > >> > > > > > >>>>> On Sun, Oct 29, 2023 at 12:17 AM Martin >>> Desruisseaux < >>> > >> > > > > > >>>>> martin.desruisse...@geomatys.com> wrote: >>> > >> > > > > > >>>>> >>> > >> > > > > > >>>>>> Le 2023-10-28 à 22 h 54, Tamás Cservenák a écrit : >>> > >> > > > > > >>>>>> >>> > >> > > > > > >>>>>>> I still see these just as new dependency types: >>> > >> "module", >>> > >> > > > > "agent", >>> > >> > > > > > >>>>>>> "doclet", and so on. >>> > >> > > > > > >>>>>>> >>> > >> > > > > > >>>>>> Does "dependency type" means the <type> element >>> inside >>> > >> > > > > <dependency>? >>> > >> > > > > > >>>>>> If >>> > >> > > > > > >>>>>> yes, then specifying a different type causes Maven >>> to >>> > >> > > download a >>> > >> > > > > > >>>>>> different JAR, without changing the kind of path >>> (class >>> > >> path >>> > >> > > > > versus >>> > >> > > > > > >>>>>> module path) where the JAR is put. The proposed >>> <usage> >>> > >> > > element >>> > >> > > > > (or >>> > >> > > > > > >>>>>> whatever equivalent alternatives) has the opposite >>> > >> semantic: >>> > >> > > it >>> > >> > > > > does >>> > >> > > > > > >>>>>> not >>> > >> > > > > > >>>>>> change the JAR to download, but put it on a >>> different >>> > >> kind >>> > >> > of >>> > >> > > > > path. >>> > >> > > > > > >>>>>> >>> > >> > > > > > >>>>>> Martin >>> > >> > > > > > >>>>>> >>> > >> > > > > > >>>>>> >>> > >> > > > > > >>>>>> >>> > >> > > > > > >>>>>> >>> > >> > > > > > >>> > >> > >>> --------------------------------------------------------------------- >>> > >> > > > > > >>>>>> To unsubscribe, e-mail: >>> > dev-unsubscr...@maven.apache.org >>> > >> > > > > > >>>>>> For additional commands, e-mail: >>> > >> dev-h...@maven.apache.org >>> > >> > > > > > >>>>>> >>> > >> > > > > > >>>>>> >>> > >> > > > > > > >>> > >> > > > > > >>> > >> > > > > > >>> > >> > >>> --------------------------------------------------------------------- >>> > >> > > > > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>> > >> > > > > > For additional commands, e-mail: >>> dev-h...@maven.apache.org >>> > >> > > > > > >>> > >> > > > > > >>> > >> > > > > >>> > >> > > > >>> > >> > > >>> > >> > >>> > >> >>> > > >>> > >>> >>
