Re: Question on shading and missing dependencies

[Francois Marot]

Hello Lars,

ignoring your second email, I felt like it is normal for the pom to ignore
the shaded dependencies. This is how Maven works.
For me, it should be the job of a SBOM (CycloneDX format or SPDX for
exemple) to keep the information of what the shaded jar contains.
This is the role of SBOMs. So in an ideal world, the shade plugin would
produce an SBOM to keep track of what is included in the shaded jar.
In an ideal world, SBOMs would also be kept for each Maven jars in the
repositories so they coule be aggregated by the shade plugins.

After reading your second mail, "keepDependenciesWithProvidedScope" may be
the best easy option you have right now, in a not-ideal world.

François

Le lun. 13 mai 2024 à 17:07, Lars Francke  a écrit :

> To add to my question I just found that the shade plugin has an option
> called "keepDependenciesWithProvidedScope"[1] which might have helped
> here.
>
> [1] <
> https://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#keepDependenciesWithProvidedScope
> >
>
>
> On Mon, May 13, 2024 at 4:50 PM Lars Francke 
> wrote:
> >
> > Hi,
> >
> > we're hunting vulnerabilities in our dependency tree and I have a
> > question that came up while doing so.
> >
> > We are using HBase (I'm a committer there as well) and HBase has (had)
> > a dependency on the now retired HTrace:
> >
> > 
> >   org.apache.htrace
> >   htrace-core4
> > 
> >
> > HTrace in version 4.2.0-incubating has a dependency on
> > jackson-databind 2.4.0[1, 2] but it also uses the shade plugin to
> > relocate all of its dependencies[3].
> > The published POM of HTrace contains no trace of these dependencies[4].
> >
> > A vulnerability scanner like Trivy[5] does find the code via the
> > META-INF/maven/ files but if HTrace is included via various levels
> > (e.g. Phoenix -> HBase -> HTrace) it is very hard to follow along and
> > find the actual place a dependency comes from.
> >
> > I also tried the Maven CycloneDX generator and that also does not list
> > jackson-databind as a dependency.
> >
> > My question: Is this expected? How can I build an accurate dependency
> > tree that includes all dependencies?
> >
> > I feel like I must be doing something obvious wrong.
> >
> > Thank you for your help.
> >
> > Cheers,
> > Lars
> >
> >
> > [1] <
> https://github.com/apache/incubator-retired-htrace/blob/2ce9d3b25a49d371a7b48e389b56d50a0164c8a0/pom.xml#L308-L312
> >
> >
> > [2] <
> https://github.com/apache/incubator-retired-htrace/blob/2ce9d3b25a49d371a7b48e389b56d50a0164c8a0/htrace-core4/pom.xml#L98-L101
> >
> >
> > [3] <
> https://github.com/apache/incubator-retired-htrace/blob/2ce9d3b25a49d371a7b48e389b56d50a0164c8a0/htrace-core4/pom.xml#L52-L59
> >
> >
> > [4] <
> https://repo1.maven.org/maven2/org/apache/htrace/htrace-core4/4.2.0-incubating/htrace-core4-4.2.0-incubating.pom
> >
> >
> > [5] 
>
> -
> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
> For additional commands, e-mail: users-h...@maven.apache.org
>
>

Re: Will this potentially break in future releases ?

[Francois Marot]

Hello Tommy, Lasse and all,
as Lasse suggests it *may* be a good fit for classifiers. But I want to
emphasize that to my knowledge, the miscellaneous artifacts with different
classifiers sharing the same 'main' artifact HAVE TO BE INSTALLED OR
DEPLOYED AT THE SAME TIME. So if you deploy your artifacts somewhere, it
may be a problem.

If you keep on relying on the version (or artifactId) to differentiate the
artifacts, you may also face problems in case you deploy your artifact on a
repository to be consumed by others: you will currently produce an invalid
pom.xml which is unusable by others. You have to use the flatten maven
plugin and configure your build correctly to produce a "patched" pom where
the version is replaced. You may find more info here
https://maven.apache.org/maven-ci-friendly.html (Maven 3) or in a recent
blog post from Karl Heinz Marbaise here for Maven 4 (which do not require
flattening the pom):
https://blog.soebes.io/posts/2024/03/2024-03-31-maven-4-part-i/

Hope it helps

Francois

Le dim. 7 avr. 2024 à 19:28, Lasse Lindqvist 
a écrit :

> Even if putting the bytecode version in the version works, you should
> consider using classifier field instead. See for example
> https://www.baeldung.com/maven-artifact-classifiers
>
> su 7. huhtik. 2024 klo 20.22 Tommy Svensson (to...@natusoft.se) kirjoitti:
> >
> > These days I'm *only* coding in Groovy since it is such a nice, painless
> language and most of my GitHub code is in Groovy.
> >
> > Groovy does a lot different than Java, one such difference is that you
> can now select what bytecode level to produce when building, this
> independent from the source. That is, the same source can be built for many
> different byte code levels without any changes in source code. You just
> select at what bytecode level you want the binary to be.
> >
> > I want to be able to build binaries for multiple bytecode levels and
> thus I'm doing this:
> >
> > http://maven.apache.org/POM/4.0.0;
> >  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
> >  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> http://maven.apache.org/xsd/maven-4.0.0.xsd;>
> > 4.0.0
> >
> > 
> > se.natusoft.seagull
> > Seagull-APIs
> > 1.0.0
> > 
> >
> > Seagull-Internal-APIs
> > ${parent.version}_${bytecode.version}
> > This actually works! I do however get warnings about using variable
> expressions in version tag.
> >
> > Will this feature potentially go away in future versions of maven ?
> >
> > My goal is to be able to produce same code at different byte-code
> levels. By appending byte-code level in version I can produce multiple
> artifacts for different byte-code levels, in a relatively easy way. But if
> using variables in  tag goes away then it becomes much more
> difficult. That it works now I see as a very nice and flexible feature.
> >
> > BR / Tommy
> > __
> > Tommy Svensson
> > to...@natusoft.se
> >
> >
>
> -
> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
> For additional commands, e-mail: users-h...@maven.apache.org
>
>

Re: [DISCUSS] Maven Dependency Plugin

[Francois Marot]

Thanks Tamas for all your work. I'll sure have a look (but not right now as
I'm in a train station on a phone). Just to mention a feature I missed
yesterday in m-d-p: ability to filter on classifiers including *wildcards*.
Because I have dependencies with this kind of classifiers: natives-win,
natives-linux,  natives-x86, natives-amd64 and so on...

Wildcard are sometimes a feature I miss in many plugins.
Thanks again for the work, wildcards or not !

Francois

Le mar. 26 mars 2024, 17:58, Tamás Cservenák  a écrit :

> Just for those brave... if you toy with it.
>
> The "copy" and "copy-transitive" CLI commands and Mojos have "targetSpec"
> parameters, that are parsed into ArtifactSink here:
>
> https://github.com/maveniverse/toolbox/blob/main/shared/src/main/java/eu/maveniverse/maven/toolbox/shared/internal/ToolboxCommandoImpl.java#L251-L288
>
> So, the copy (and copy-transitive) resolve (transitively) and push the
> results into an ArtifactSink.
>
> Sink spec can be:
> "foo" -> implied "flat:foo"
> "flat:" -> a "flat directory" to copy artifacts to, like "flat:lib"
> (is resolved from basedir)
> "repository:" -> creates a repo that can be used as "file://" uri or
> published via HTTP (is resolved from basedir)
> "install:" -> will install them into local repository (session
> default in no path specified), ie. good to prep a local repo -- TBD 
> param
> "deploy:id::url" -> will deploy them to Remote Repo
> "purge:" -> will purge them from local repository (session default in
> no path specified) -- TBD  param
>
> So, following command:
>
> copy -DdepSpec="any()" -DtargetSpec="purge:"
>
> actually does the purge from the local repository. Same stands for gav-copy
> (and gav-copy-transitively), but they do not use MavenProject to
> "contextualize" but need to be told explicitly what to resolve...
>
>
> On Tue, Mar 26, 2024 at 5:34 PM Tamás Cservenák 
> wrote:
>
> > Howdy,
> >
> > Yes, m-dep-p is under maintained, it actually would need a rewrite as it
> > still uses MAT (and many other Maven2 archaic stuff) internally.
> > Hence, it will fail if used with 3.9+ features like "split repository"
> and
> > is suboptimal in many areas.
> >
> > Toolbox 0.1.0 released, btw:
> >
> > jbang toolbox@maveniverse
> >
> > or
> >
> > mvn mvn eu.maveniverse.maven.plugins:toolbox:0.1.0:gav-repl
> >
> > to enter REPL (same as in MIMA CLI was).
> >
> > T
> >
> > On Tue, Mar 26, 2024 at 4:51 PM Greg Chabala 
> > wrote:
> >
> >> Hello Tamás,
> >>
> >> For context, what are the tensions that you're trying to solve here?
> >>
> >> Is m-dependency-p too big/getting unmaintainable/becoming a kitchen
> sink?
> >>
> >> Do some goals feel like a bad fit?
> >>
> >> Are you thinking of breaking it up or replacing it?
> >>
> >> Greg
> >>
> >> On Tue, Mar 26, 2024 at 8:52 AM Tamás Cservenák 
> >> wrote:
> >>
> >> > Howdy,
> >> >
> >> > just to not let this discussion die off. Let me show a take on a "how
> >> > modern Maven plugin should look like" (that targets m-dependency-p
> >> goals,
> >> > sans analyze and some others) could look like:
> >> > https://github.com/maveniverse/toolbox
> >> >
> >> > The "unpack" related goals are missing, not yet done, but there are
> >> already
> >> > 33 Mojos in there. Most Mojos that are "gav-" prefixed are totally
> same
> >> as
> >> > CLI commands, and they do NOT require Project, are meant to be "ad
> hoc"
> >> > invoked.
> >> > The non-"gav-"-prefixed mojos use MavenProject instead to
> >> "contextualize"
> >> > themselves (so they work with the Project, and the "plugin-" prefixed
> >> ones
> >> > with Project defined plugins).
> >> >
> >> > Note1: not yet released (is not on Central), if you want to test drive
> >> it,
> >> > build it locally).
> >> > Note2: the "toolbox" is Maven Plugin and a CLI at the same time (the
> CLI
> >> > uber is toolbox-1.0.0-SNAPSHOT-cli.jar)
> >> > Note3: some of the missing goals mentioned on this thread are
> >> implemented
> >> >
> >> > Thanks
> >> > T
> >> >
> >>
> >
>

Re: [DISCUSS] Maven Dependency Plugin

[Francois Marot]

Hey !
First of all, for everyone asking more info to understand where the pom
values come from and how they are computed, I think a very useful command
is:
**
mvn org.apache.maven.plugins:maven-help-plugin:3.4.0:effective-pom
-Dverbose=true
**
It will display your effective pom but also will tell you in comments for
each value where (which pom) it comes from.

Regarding the dependency plugin, what I use very frequently:
* copy-dependencies (including all the options)
* tree

What may be usefull to me
* analyze-*
* build-classpath
* dependency:unpack-dependencies (see below)


For the record, I forked/rewrote a plugin kinda similar to the
unpack-dependencies specifically aimed at unzipping zip or tar.gz
containing dlls being identified by the classifier "natives-win" or
"natives-linux": https://github.com/fmarot/nativedependencies-maven
The main selling point to this plugin is its speed in a multi-module
project (I have 150 modules needing native dependencies being unzipped in
their PATH for the tests) because it keeps record of the unpacked
dependencies and will skip unpacking the same dependency again and again
when you want all the dlls being unpacked at
${session.executionRootDirectory}\target\natives for example.

Regards,
François

Le jeu. 21 mars 2024 à 17:06, Tamás Cservenák  a
écrit :

> Howdy,
>
> I'd would be interested in how users and devs are using
> maven-dependency-plugin:
> https://maven.apache.org/plugins/maven-dependency-plugin/
>
> I collected some basic questions I'd like to have answered (but feel free
> to add more info!):
> - which goals are "must have" for you
> - which goals are "I never touched" for you (or, "I really don't need" or
> "never used" or "shrug")
> - what is missing?
>
> Thanks
> T
>

Re: Offloading projects cli flag into profile

[Francois Marot]

Hello Mantas,

I think you did not explain what you are trying to achieve. Or at least I
did not really understand.
Are you trying to lower the build time ?
If yes, then I would advise to use a Maven property for the configuration
in the root pom
Then most of your build could be run with -DskipRepackage=true and only
some could use -DskipRepackage=false




org.springframework.boot
spring-boot-maven-plugin
x.y.z


repackage

repackage


${skipRepackage}






François


Le jeu. 21 mars 2024 à 13:43, Mantas Gridinas  a
écrit :

> I have several spring boot applications in my reactor that need to be
> repackaged before deployment. Only those spring boot modules have reference
> to the spring boot plugin.
>
> My current approach is to build the entire reactor (via install goal), and
> later run repackaging goal only on particular spring boot modules like
> follows
>
> mvn install
> mvn package spring-boot:repackage --projects
> springboot1,springboot2,springboot3
>
> Can the projects list be offloaded into profile configuration? I have a
> feeling the spring boot entrypoints will only proliferate in the future and
> would rather modify that list via pom rather than the multiple
> pipelines/cicd applications. I suppose workaround would be to manage a
> second pom which only lists those modules but then things might get hairy
> with parent references.
>

Re: How does maven resolve inter-module dependencies in a multi-module build?

[Francois Marot]

to add on Joseph answer: if you are in the B directory, then only B is in
the reactor (more or less meaning "the list of project being built by this
Maven invocation").
My rule of thumb is to never do that and always run from the root. If you
want to speedup the build and you know that only B has been modified, you
can use maven options such as:

- mvn test -pl :B // it will only build/test the B module
- mvn test -pl :B -am// it will build B and all the modules B
depends onto
- mvn test -rf :B// it resumes the build from B in case B's
tests failed , so it will also build modules after B

you can note that I add a semi-column before the module name because in
case the module is a sub-module (or sub-sub-sub...module), not directly at
the parent's root, then it is required.
Hope this helps


Le jeu. 15 févr. 2024 à 20:55, Joseph Leonard <
joseph.leon...@alfasystems.com> a écrit :

> Builds in the reactor are always favoured over the m2 repo because the
> builds in the reactor will be up-to-date whereas the jar in the repo may be
> out of date.
>
> If you run mvn compile in directory B then you will not be running a multi
> module build - and therefore module A will not be in the reactor. This is
> why the m2 repo will always be used for module A in this scenario.
>
> Joe
>
> -Original Message-
> From: Siddharth Jain 
> Sent: Thursday, February 15, 2024 7:50 PM
> To: Maven Users List 
> Subject: Re: How does maven resolve inter-module dependencies in a
> multi-module build?
>
> External Email: Please be vigilant and check the contents and source for
> signs of malicious activity.
>
> thanks Joe. but then if classes are available both in the target directory
> of module A as well as a jar file in M2 repository which takes precedence?
>
> also i have noticed that while running mvn compile from the root works,
> running mvn compile from the directory of B does not pick up classes from
> A's target directory. it only picks up from M2 repo in that case.
>
> On Thu, Feb 15, 2024 at 11:28 AM Joseph Leonard <
> joseph.leon...@alfasystems.com> wrote:
>
> > Hi Sid,
> > It will resolve the classes directory of module A that will have been
> > populated during module A's 'compile' build.
> > Joe
> >
> > On 2024/02/15 17:50:44 Siddharth Jain wrote:
> > > Hello,
> > >
> > > I am working on a multi-module Maven build. e.g., I have a root
> > > directory containing 3 sub-projects A, B, C and a parent pom defined
> > > in the root directory. I notice that I can run mvn compile from the
> > > root directory
> > and
> > > it will build the 3 projects. The projects may have
> > > inter-dependencies e.g., B depends on A and let's say C depends on
> both A and B.
> > >
> > > My question is while building B how does maven locate the compiled
> > > code
> > of
> > > A (the dependency) since maven compile by itself does not install
> > > the
> > built
> > > artifact into M2 repository?
> > >
> > > Sid
> > >
> >
>

Re: use 'mvn insall:install-file' to install jar file with pom.xml file,is there a way to produce pom.xml without '-DpomFile=path-to-pom'

[Francois Marot]

The pom.xml will be automatically produced even without the '-DpomFile'
argument. But it will be very minimalist. PAssing your own pom allows to be
explicit regarding the dependencies, the licences etc...
Regards

Le dim. 4 févr. 2024 à 11:52, sandisks...@gmail.com 
a écrit :

> use 'mvn insall:install-file' to install jar file with pom.xml file,is
> there a way to produce pom.xml without '-DpomFile=path-to-pom'
>
>
>
> sandisks...@gmail.com
>

How to know which plugin/goal is frozen in multithreaded build ?

[Francois Marot]

Hello all,

I have a large build that completes normally in single threaded build but
freezes in multithreaded builds. Problem is I do not know which module and
which plugin is frozen. The last logs that get displayed are the classic
end log for a given module and the logs always stop there. So I believe
another module is still underway, stuck, and prevents the build to go
further. If there were entry/exit logs for each module and each plugin, I
could parse the logs, match each entry/exit, and find the last missing
exit. That would point me to the culprit...
But to my knowledge there are no such entry/exit logs...

I'will try to use mvnDebug command and plug a debugger but I do not even
know what I'm looking for ! Any idea ?

Have a good day
François

Re: Random missing exclusions in dependency-reduced-pom when run in //

[Francois Marot]

I think Estelle's mail lacked the context so I added it below. She made a
reproducer for a probable bug regarding missing exclusions from the
dependency-reduced-pom.xml in recent versions of the Shade plugin when run
in parrallel. Thanks Estelle !

Francois

Le ven. 19 janv. 2024 à 18:37, Estelle Bremont 
a écrit :

> Hello !
> I work with François. Here's a quite simple example reproducing the problem
> : https://github.com/stl543/shadeMT
> It has a parent pom so the 4 child modules have strictly the same
> configuration apart from their artifactId.
>
> The following command will give you results varying between 12, 16 and 20
> showing that  are sometimes missing from the generated
> dependency-reduced-pom.xml. Usually all the poms are generated the same
> (which is expected) but sometimes some have exclusions that others do not.
>
> mvn clean package -T 1C && find . -name "dependency-reduced-pom.xml" -exec
> bash -c 'cat $0 | grep exclu | wc -l ' {} \;
>
>
>
> - shade 3.5.1 & 3.4.1 : have the problem of missing exclusions
> - shade ranging from 3.2.1 to 3.4.0 have freeze problems (known bug)
> - shade 3.2.0 : ok
> - shade 3.1.0 : ok
>
> Do you think I should open an issue directly with the shade plugin ?
> Best regards,
> Estelle
>

-- Forwarded message -
De : Alexander Kriegisch 
Date: ven. 12 janv. 2024 à 03:21
Subject: Re: Random missing exclusions in dependency-reduced-pom when run
in //
To: 


Hello François.

I am not a committer in any Apache project, but I have contributed to
Shade here and there.

A reproducer would certainly help, so please post GitHub link, as soon
as you have one. If it is clearly reproducible, it might also be wort
opening an issue to have something to commit on, if anyone feels like
creating a PR.

Cheers
-- 
Alexander Kriegisch
https://scrum-master.de


Francois Marot schrieb am 11.01.2024 16:16 (GMT +07:00):

> Hello,
>
> First let me wish the Maven community a happy new year.
> I recently faced a problem with the shade plugin generating randomly
> different dependency-reduced-pom.xml.
> The difference lies in exclusions defined in miscellaneous dependencies
> randomly being missing in the pom.
> This part of the build builds multiple (4) project in // (-T 1C) all using
> the shade plugin to generate a dependency-reduced-pom.xml.
> All my tests are done with the latest maven-shade-plugin 3.5.1
> I think there is a concurrency issue somewhere and I think it is in Maven
> model being manipulated by the shade plugin.
>
> With Maven 3.9.4 I was able to reproduce the problem only once. With 3.9.5
> & 3.9.6 it started to be a lot more reproducible.
> Could the Shade plugin use Maven model in a non thread safe way ? Any idea
> welcome !
> (I'm sorry and haven't taken the time to dev a SCCE(http://sscce.org/) yet
> but I may try if no one has an idea of what is happening)
>
> Regards
> Francois
>

Random missing exclusions in dependency-reduced-pom when run in //

[Francois Marot]

Hello,

First let me wish the Maven community a happy new year.
I recently faced a problem with the shade plugin generating randomly
different dependency-reduced-pom.xml.
The difference lies in exclusions defined in miscellaneous dependencies
randomly being missing in the pom.
This part of the build builds multiple (4) project in // (-T 1C) all using
the shade plugin to generate a dependency-reduced-pom.xml.
All my tests are done with the latest maven-shade-plugin 3.5.1
I think there is a concurrency issue somewhere and I think it is in Maven
model being manipulated by the shade plugin.

With Maven 3.9.4 I was able to reproduce the problem only once. With 3.9.5
& 3.9.6 it started to be a lot more reproducible.
Could the Shade plugin use Maven model in a non thread safe way ? Any idea
welcome !
(I'm sorry and haven't taken the time to dev a SCCE(http://sscce.org/) yet
but I may try if no one has an idea of what is happening)

Regards
Francois

Re: How to pass java.library.path to mvn exec?

[Francois Marot]

for the record, I find it simpler to hardcode it in the pom like this:
-Djava.library.path=...

Here is my pom:



org.codehaus.mojo

exec-maven-plugin





runUnobfuscated



java





${session.executionRootDirectory}/${buildDirectory}/natives/:${java.home}/lib/amd64/



${PATH};${session.executionRootDirectory}/${buildDirectory}/natives/;${java.home}/lib/amd64/

0



true

true



-Djava.library.path
=${session.executionRootDirectory}/${buildDirectory}/natives/;${session.executionRootDirectory}/${buildDirectory}/natives//jre/bin

-Ddev=true

-Duser.language=en

-XX:MaxDirectMemorySize=1024m

[...]

Le sam. 6 janv. 2024 à 16:20, Neil Aggarwal  a
écrit :

> > As far as I get you should pass java.library.path in the java executable
> > arguments
>
> Yes, that worked, thank you!
>
> Thank you,
>Neil
>
> --
> Neil Aggarwal, 972-834-1565, http://propfinancing.com
> We offer 30 year loans on single family houses!
>
> -
> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
> For additional commands, e-mail: users-h...@maven.apache.org
>
>

Re: Configuring plugins by composition rather than inheritance

[Francois Marot]

For the record, Spring Boot has no hard requirement of using a 'parent'
pom. It is just a bit simpler and in most of the examples online.
But you can simply set the scope import in your dependencyManagement. This
works.





org.springframework.boot

spring-boot-dependencies

${spring.boot.version}

pom

import




Hope it helps



Le mar. 5 déc. 2023 à 12:51, Bernd Eckenfels  a
écrit :

> Hello,
>
> I think that’s a long-standing restriction of mavens Pom-model, not sure
> it can easily be solved. Spring boot (and other frameworks) did not make
> the situation easier with their parent requirement (but then again there is
> not much alternatives).
>
> From my experience it seems a good idea to get away from a strict „simple
> company root“ requirement. (Due to release cycles it doesn’t do much for
> unifying build jobs anyway).
>
> Maven could improve with importing build/plugin sections like BOMs. I
> haven’t seen much talk about that in recent 4.0 discussions,
>
> Having said that a consumer Pom goes a good distance to make the parents
> less relevant. (Especially licensing/orga matters)
>
> Gruß
> Bernd
>
> Mantas Gridinas wrote on 5. Dec 2023 09:17 (GMT +01:00):
>
> > In my current project i'm working we already have a parent POM that I
> > should be inheriting from, but when trying to integrate the spring boot I
> > come into an issue that I should inherit their super pom instead of using
> > our own. For dependencies its pretty simple - you use the import scope,
> > but
> > what about plugins? I'm seeing some stackoverflow posts about the sameish
> > issue from 11 years ago that the suggestion was to use tiles plugin. Was
> > there any movement regarding this in the mean time?
> >
> > As a workaround I currently have a dedicated springboot runtime module
> > that
> > includes a single entry point with respective annotations to start the
> > application, and it directly inherits the spring boot super pom while
> > importing our dependencies via the depencendy management block but having
> > a
> > submodule not inherit a parent module feels weird
> >
>
>
> Gruß
> Bernd
> —
> https://bernd.eckenfels.net
>
> -
> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
> For additional commands, e-mail: users-h...@maven.apache.org
>
>

Re: what means "The following artifacts could not be resolved: org.dom4j:dom4j:jar:2.1.3 (present, but unavailable)"

[Francois Marot]

Hello Arno and Tamas

and thank you for the detailed and very interesting explanation.
One question arises though: is there any way to make Maven ignore the
origin ?

Have a good day
François


Le lun. 6 nov. 2023 à 13:20, Tamás Cservenák  a écrit :

> Howdy,
>
> The Maven local repository contains two kinds of artifacts lumped together:
> - cached ones from remote
> - locally built and installed
>
> This message matters the cached ones: "present but unavailable" means
> following:
> - file IS present (so it was cached)
> - but is unavailable, as it was cached as part of some OTHER remote
> repository than that exists in current execution
>
> Maven3 uses this implementation of local repository
>
> https://maven.apache.org/resolver/apidocs/org/eclipse/aether/internal/impl/EnhancedLocalRepositoryManagerFactory.html
>
> And check out the javadoc it: it tracks the "origin" (basically remote
> repository ID) of cached artifacts, and "emulates" physically separated
> caches (same can be achieved "for real" with split local repository).
>
> In short: the repository ID that was stored as artifact origin is NOT
> available in your subsequent build, hence, Maven3 does not use it (is
> unavailable, but present physically in cache/local repository).
>
> HTH
> T
>
> On Mon, Nov 6, 2023 at 1:10 PM Arno Schatz  wrote:
>
> > Hi,
> >
> > I am trying to use the offline mode (mvn -o clean install) and getting
> the
> > error message
> > "The following artifacts could not be resolved: org.dom4j:dom4j:jar:2.1.3
> > (present, but unavailable)"
> > for very many dependencies.
> >
> > My main computer and a VM share the same file system. The VM is connected
> > to the company's repository (artefactory)
> > through VPN. On the VM, I checkout the project and successfully build it
> > and successfully ran "mvn dependency:go-offline".
> >
> > The I switched to my main computer and tried "mvn -o clean install" but
> it
> > failed with the above error message. I also
> > tried to set
> >   true
> > in settings.xml in my main computer but that didnt change anything.
> > What does the "(present, but unavailable)" in the error message mean?
> >
> > thanks,
> > Arno
> >
> >
> > -
> > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
> > For additional commands, e-mail: users-h...@maven.apache.org
> >
> >
>

Re: maven-build-cache, immutable docker tags, etc

[Francois Marot]

You're right...
As I see it, you are forced to have a different version number for
different branches. As such, the build cache may not be effective at all
because I imagine the version number is one of the first input considered
to determine if the cache can be reused or not.

Disclaimer: I've never used the build-cache plugin (yet, but I plan to) so
I may be wrong. Please keep us updated on your plan (I'm interested in the
result)

Le dim. 24 sept. 2023, 15:14, Ben Tatham 
a écrit :

> Thanks, Francois.  That would have been great for our original problem, but
> how I can I leverage that for build cache issues?
>
> On Sun, Sept 24, 2023, 5:14 a.m. Francois Marot 
> wrote:
>
> > Just a suggestion: you could use ci-friendly Maven to pass in a version
> > depending on the commit Id/sha1 on the command line. Like mvn
> > -Drevision=
> > More info:
> > https://maven.apache.org/maven-ci-friendly.html
> >
> > Regards
> >
> > Le jeu. 21 sept. 2023, 15:49, Ben Tatham  > .invalid>
> > a écrit :
> >
> > > Hi,
> > > I'm loving the ideas in the maven-build-cache, but I'm running into
> some
> > > circular build issues that are blocking us from using it at the moment.
> > >
> > > In many of our projects, we deploy docker images. Ideally, we would
> like
> > to
> > > keep our docker images immutable. To achieve that, we have extra maven
> > > plugins that create unique SNAPSHOT version numbers (based on jira
> issue
> > > ids, git commit hashes, and/our CI build numbers).  We do this for
> docker
> > > image immutability, but also to avoid SNAPSHOT collisions when multiple
> > > developers are working at the same time in the same project, but still
> be
> > > able to deploy the snapshot artifacts for other reasons (temporary test
> > > deployments, sharing libraries as snapshots while still in development
> > > across projects, etc).
> > >
> > > But that solution creates unique maven versions in CI for every build,
> > > which of course then contradicts using maven-build-cache, since the
> > version
> > > is always different in CI.
> > >
> > > To use maven-build-cache, we would just go back to using standard
> > -SNAPSHOT
> > > builds, but then we have to solve our original issues a different way.
> > >
> > > (Sadly, AWS ECR only allows setting immutability at the registry
> level. I
> > > looked at nexus as a docker registry, and it seems to be even higher at
> > the
> > > whole docker repo level. (We wish we could set immutability of images
> > based
> > > on the tag - ie allow -SNAPSHOT tags to mutable, but nothing else))
> > >
> > > One idea we have is to use a different docker registry for snapshot vs
> > > release builds, but since maven properties are evaluated at the
> beginning
> > > of the maven lifecycle, we cannot change the parameters sent to maven
> > > plugins that due to the docker builds once the build starts. Thus, we
> > would
> > > have to do something outside of maven to detect, eg, if the build is a
> > > snapshot and choose a different docker registry there and pass it on
> the
> > > command line. We would love this to be an all-maven solution.
> > >
> > > Does anyone else run into similar issues with maven-build-cache?  Is
> > there
> > > some weird way we can ignore the project.version altogether on the
> > > maven-build-cache (that seems very odd, for sure though)?  Other ideas?
> > >
> > > Thanks,
> > >
> > > Ben
> > >
> > > [image: -] <http://www.nanometrics.ca>
> > > [image: -] <https://twitter.com/nanometricsinc> [image: -]
> > > <
> https://www.linkedin.com/company/nanometrics-seismological-instruments/
> > >
> > > [image:
> > > -] <https://www.youtube.com/channel/UCwao286g87g7Mqj6qvnLIZw>
> > >
> > > Ben Tatham
> > > Principal Software Developer
> > > Nanometrics
> > > bentat...@nanometrics.ca
> > >
> > > --
> > > This message is intended exclusively for the individual or entity to
> > which
> > > it is addressed. This communication may contain information that is
> > > proprietary, privileged, confidential or otherwise legally exempt from
> > > disclosure. If you are not the named addressee, or have been
> > inadvertently
> > > and erroneously referenced in the address line, you are not authorized
> to
> > > read, print, retain, copy or disseminat

Re: maven-build-cache, immutable docker tags, etc

[Francois Marot]

Just a suggestion: you could use ci-friendly Maven to pass in a version
depending on the commit Id/sha1 on the command line. Like mvn
-Drevision=
More info:
https://maven.apache.org/maven-ci-friendly.html

Regards

Le jeu. 21 sept. 2023, 15:49, Ben Tatham 
a écrit :

> Hi,
> I'm loving the ideas in the maven-build-cache, but I'm running into some
> circular build issues that are blocking us from using it at the moment.
>
> In many of our projects, we deploy docker images. Ideally, we would like to
> keep our docker images immutable. To achieve that, we have extra maven
> plugins that create unique SNAPSHOT version numbers (based on jira issue
> ids, git commit hashes, and/our CI build numbers).  We do this for docker
> image immutability, but also to avoid SNAPSHOT collisions when multiple
> developers are working at the same time in the same project, but still be
> able to deploy the snapshot artifacts for other reasons (temporary test
> deployments, sharing libraries as snapshots while still in development
> across projects, etc).
>
> But that solution creates unique maven versions in CI for every build,
> which of course then contradicts using maven-build-cache, since the version
> is always different in CI.
>
> To use maven-build-cache, we would just go back to using standard -SNAPSHOT
> builds, but then we have to solve our original issues a different way.
>
> (Sadly, AWS ECR only allows setting immutability at the registry level. I
> looked at nexus as a docker registry, and it seems to be even higher at the
> whole docker repo level. (We wish we could set immutability of images based
> on the tag - ie allow -SNAPSHOT tags to mutable, but nothing else))
>
> One idea we have is to use a different docker registry for snapshot vs
> release builds, but since maven properties are evaluated at the beginning
> of the maven lifecycle, we cannot change the parameters sent to maven
> plugins that due to the docker builds once the build starts. Thus, we would
> have to do something outside of maven to detect, eg, if the build is a
> snapshot and choose a different docker registry there and pass it on the
> command line. We would love this to be an all-maven solution.
>
> Does anyone else run into similar issues with maven-build-cache?  Is there
> some weird way we can ignore the project.version altogether on the
> maven-build-cache (that seems very odd, for sure though)?  Other ideas?
>
> Thanks,
>
> Ben
>
> [image: -] 
> [image: -]  [image: -]
> 
> [image:
> -] 
>
> Ben Tatham
> Principal Software Developer
> Nanometrics
> bentat...@nanometrics.ca
>
> --
> This message is intended exclusively for the individual or entity to which
> it is addressed. This communication may contain information that is
> proprietary, privileged, confidential or otherwise legally exempt from
> disclosure. If you are not the named addressee, or have been inadvertently
> and erroneously referenced in the address line, you are not authorized to
> read, print, retain, copy or disseminate this message or any part of it.
> If
> you have received this message in error, please notify the sender
> immediately by e-mail and delete all copies of the message.
>

Re: Multi-module Maven projects and Scala-aware artifact IDs

[Francois Marot]

Hello Florent,

i'll suggest something while I know this is not exactly what you asked for.
But i'll try nevertheless, it may help :)

If you could change your requirements a bit to change the *version* instead
of the artifactId, then you could use the
ci-friendly concept in Maven (
https://maven.apache.org/maven-ci-friendly.html ). This in conjonction with
the flatten plugin
that ensures pom.xml are correctly alterned before being deployed could
help to have something quite similar to what you asked.

So my idea is to run a command similar to:

mvn deploy -Drevision=1.2.3-2.12
or
mvn deploy -Drevision=1.2.3-2.13

The consumer of your artifacts would have to update the version as well as
the artifactId, but they are usually colocated so it may be acceptable.
This is compatible with multi-module Maven projects where each child just
reference the parent pom with the ${revision} variable.
Hope this helps

François


Le lun. 12 juin 2023 à 15:21, Florent Biville  a
écrit :

> Hello,
>
> I'm working on a multi-module Maven project where part of the artifact ID
> includes the Scala version it is built against (2.12, 2.13, ...), as it is
> common with Scala projects, since Scala minor version bumps often break (or
> so I read).
>
> I'm currently trying to fix the project setup, since it relies on dynamic
> expressions in some of the module artifact IDs to achieve the above effect
> and that does not really work (and litters the build with warnings,
> rightfully so).
>
> My current approach is to script the whole thing, something like
> "./build-for-scala.sh 2.12" or "./build-for-scala-2.13". Sprinkle some sed
> and it kinda works.
>
> However, it is very fragile. Not only must the module artifact IDs change,
> but so must any reference to the parent POM and the artifact ID of each
> dependency sibling.
>
> Reading about this problem led me to
> https://stackoverflow.com/a/35023562/277128 but the solution is not very
> compelling.
>
> And of course, since that's an established project, I cannot afford to
> change the artifact IDs before the next major version.
>
> I would not even mind writing my own plugin or extension if that could
> solve this particular dynamic artifact ID resolution.
>
> Has anyone faced this before? How did you manage to solve this problem "the
> Maven way"?
>
> Best regards,
> Florent
>

Re: [External] : Re: NPE Exception with Surefire3.0.0

[Francois Marot]

You should get rif of all the dependencies and then test if your problem
has disappeared.
If yes, all good, and next time let Maven handle transitive dependencies
for you.
If no, then you have a real problem that maybe some people here will be
able to help you with.

But overriding all the dependencies is just plain impossible to maintain:
imagine the sligest incompatibility between 2 dependency might lead to
errors if you a version instead of another !
And in case of upgrading a plugin, what a huge amount of work to update all
its dependencies ! Unmanageable...

good luck



Le mar. 18 avr. 2023 à 06:13, Sirisha Alla  a
écrit :

> I am a newbee to Maven and I want to override any dependencies (along with
> versions) from parent pom.xml. Not sure if this is the right way.
>
> Thanks,
> Sirisha
>
> -Original Message-
> From: Olivier Lamy 
> Sent: Tuesday, April 18, 2023 11:29 AM
> To: Maven Users List 
> Subject: Re: [External] : Re: NPE Exception with Surefire3.0.0
>
> Hi
> Do you really need the dependencies section within the plugin section?
> just use
>   
> 
> 
> org.apache.maven.plugins
> maven-surefire-plugin
> 3.0.0
>   
> 
> 
>
>
> On Tue, 18 Apr 2023 at 15:56, Sirisha Alla 
> wrote:
> >
> > I am using the maven version 3.8.7. pom.xml has other internal module
> dependencies and below is the snippet from pom.xml for the junit tests
> added.
> >
> > 
> > org.mockito
> > mockito-core
> > 4.11.0
> >  test
> > 
> > 
> > org.junit.jupiter
> > junit-jupiter-api
> > 5.9.2
> > test
> > 
> >   
> >
> >   
> > 
> > 
> > org.apache.maven.plugins
> > maven-surefire-plugin
> > 3.0.0
> > 
> > 
> > org.apache.maven.surefire
> > maven-surefire-common
> > 3.0.0
> > 
> > 
> > org.apache.maven.surefire
> > surefire-booter
> > 3.0.0
> > 
> > 
> > org.apache.maven.surefire
> > surefire-logger-api
> > 3.0.0
> > 
> > 
> > org.apache.maven.surefire
> > surefire-api
> > 3.0.0
> > 
> > 
> > org.apache.maven.surefire
> > surefire-extensions-api
> > 3.0.0
> > 
> > 
> > org.codehaus.plexus
> > plexus-java
> > 1.1.2
> > 
> > 
> > org.apache.maven.surefire
> > surefire-shared-utils
> > 3.0.0
> > 
> > 
> > org.apache.maven.shared
> >
>  maven-common-artifact-filters
> > 3.3.2
> > 
> > 
> > org.eclipse.aether
> > aether-util
> > 1.1.0
> > 
> > 
> > org.apache.maven.surefire
> > surefire-junit-platform
> > 3.0.0
> > 
> > 
> > org.junit.jupiter
> > junit-jupiter-engine
> > 5.9.2
> > runtime
> > 
> > 
> > org.junit.platform
> > junit-platform-engine
> > 1.9.2
> > runtime
> > 
> >
> > 
> > 
> > 
> >   
> >
> > Thanks,
> > Sirisha
> >
> > -Original Message-
> > From: Benjamin Marwell 
> > Sent: Tuesday, April 18, 2023 11:16 AM
> > To: Maven Users List 
> > Subject: [External] : Re: NPE Exception with Surefire3.0.0
> >
> > Which version of Maven are you using?
> > Can you give us a relevant snippet of your pom.xml to reproduce the
> issue?
> >
> >
> >
> > On Mon, 17 Apr 2023, 17:07 Sirisha Alla, 
> wrote:
> >
> > > Hi All,
> > >
> > > I am using surefire 3.0.0 with Junit 5 (Jupiter-engine). However, I
> > > am stumbling in to the Null Pointer Exception and could not trace the
> issue.
> > > Has anyone faced this issue or any pointers that can help me proceed?
> > > I am using JDK1.8.
> > >
> > > Caused by: java.lang.NullPointerException
> > > at
> > > org.apache.maven.plugin.surefire.AbstractSurefireMojo.hasGroupArtifa
> > > ct
> > > Id
> > > (AbstractSurefireMojo.java:1771)
> > > at
> > > 

Re: Tricky dependencies in multi module projects

[Francois Marot]

> I do not understand what you are trying to explain here.

Sorry I was not clear at all ! I was trying to say that in a multimodule
build, if you have a module shaded with the shade plugin, my rule of thumb
is to NEVER have another module listing this shaded module in its
dependencies.
Said otherwise, in the same reactor, the shaded module must be a dependency
of no other module.
This is because Maven computes the dependency graph at startup, when it has
no idea of "shading".




*- - - - -François Marot*



Le dim. 10 juil. 2022 à 11:56, Niels Basjes  a écrit :

> Hi,
>
> On Fri, Jul 8, 2022 at 5:43 PM Francois Marot 
> wrote:
>
> > I believe you used the shade maven plugin or equivalent.
>
>
> Yes
>
>
> > I faced this
> > problem multiple times and to my knowledge, there is no "good" solution.
> >
>
> I was afraid of that.
>
>
> > The problem is that Maven computes the dependency graph for all modules
> of
> > the reactor right at the start. But when the shade plugin integrates
> ANTLR
> > inside your library, this library should not depend onto ANTLR anymore.
> But
> > because the dependency graph was computed at the start,
> > other modules of the reactor depending onto your library still depend
> onto
> > ANTLR even if there is already a shaded version of ANTLR inside your
> > library.
> >
>
> Yes, I was hoping there is a simple way to detach a directory from the main
> build reactor and still have it built.
>
> So in the end, in the same reactor execution, module MUST NOT depend onto a
> > shaded module. They must be considered as 'final' modules.
> >
>
> I do not understand what you are trying to explain here.
>
> Niels
>

Re: Tricky dependencies in multi module projects

[Francois Marot]

Hello Niels,

I believe you used the shade maven plugin or equivalent. I faced this
problem multiple times and to my knowledge, there is no "good" solution.
The problem is that Maven computes the dependency graph for all modules of
the reactor right at the start. But when the shade plugin integrates ANTLR
inside your library, this library should not depend onto ANTLR anymore. But
because the dependency graph was computed at the start,
other modules of the reactor depending onto your library still depend onto
ANTLR even if there is already a shaded version of ANTLR inside your
library.

So in the end, in the same reactor execution, module MUST NOT depend onto a
shaded module. They must be considered as 'final' modules.

I hope this may help
François







*- - - - -François Marot06 50 91 96 38*



Le ven. 8 juil. 2022 à 16:51, Niels Basjes  a écrit :

> Hi,
>
> I have a library project that uses dependencies that are prone to cause
> conflicts when another project wants to use my software.
> Antlr4 is a common example because it enforces the versions of the
> generated code and the runtime to be an exact match.
>
> In my maven project I have a main library and to avoid conflicts in
> downstream applications the antlr4-runtime has been shaded and relocated to
> a different package name.
> Using this modified jar in any project works as expected.
>
> When I want to use this in a different maven module of the same project it
> does not work like that.
> In my multi module maven project I found that the other modules (UDFs, demo
> webservlet, etc)  use the unshaded variant of the library, which sometimes
> causes conflicts.
> I think this is how the maven reactor is intended to work.
>
> Essentially I think I want to have the option to make a specific module of
> my project be built as-if it is an external project (i.e. "outside" the
> reactor).
>
> So far I have only found the invoker plugin to be a way to make this
> happen.
>
> What is the recommended way to handle this?
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes
>

Re: Maven Reactor and plugin "local" dependencies

[Francois Marot]

Hello Florent,

I have had this problem a few years ago, and from what I can remember it is
the expected behavior. Or at least this is the way it is right now.
Sorry, It is from memory and I do not have any link to share nor
explanation about it, but at least you know you are not alone !

Regards

François



Le jeu. 5 mai 2022 à 15:11, Florent Biville  a
écrit :

> Hello everyone,
>
> TL;DR? https://github.com/fbiville/mvn-plugin-dependency-reactor-issue
>
> I recently noticed that, within the big multi-module project of my company,
> a specific module is listed as a dependency to plugins such as Maven
> Checkstyle plugin or com.mycila:license-maven-plugin.
>
> The problem occurs when I try one of these plugins against another specific
> module.
> Maven CLI will basically complain that it cannot resolve the module needed
> by the plugin and fail, even if I explicitly list that module in the
> project list.
>
> That occurs both with Maven 3 and 4.
>
> The obvious workaround is to install the required module first and then run
> the plugin but I'm trying to get rid of the need for `mvn install`
> entirely.
>
> Is there a particular reason plugin dependencies are not considered by
> Maven Reactor?
> Is module-as-plugin-dependency a Maven antipattern?
>
> Thanks a lot for your help!
> Florent
>

Re: Help with Shading on ActiveMQ Artemis

[Francois Marot]

Hello Clebert

I hope my comment will help you.
In my experience, using the shade plugin is tricky because the rule of
thumb I came up with is: "your shaded artifacts must be the last artifact
to be built in your multi modules Maven project".
Which can read "you must not depend onto your shaded artifact if shading
happens in the same Maven execution (or Maven reactor)".
You have to have a first execution of Maven to install/deploy your shaded
artifact, and then a second execution can depend onto this shaded artifact.
So you must have 2 different project trees.

Why so much trouble ?
because Maven computes the dependency trees for each module right at the
start of the build, whereas the Shade plugin happens to mess with the
dependency tree in the middle of Maven's execution. Shade plugin is a hack
and therefore extreme care must be taken in its use.
At least that's my understanding after using it for years and having been
bitten numerous times.

Hope it helps




*- - - - -François Marot*



Le mer. 17 nov. 2021 à 15:07, Clebert Suconic  a
écrit :

> Right.. This should happen by default. and it will only work if I'm
> using the shaded jar outside of my project. However if I use it within
> my project it will not work at all.
>
>
> I have created a minimal version of my issue on this github project:
>
>
> https://github.com/clebertsuconic/clebert-shade-issue
>
>
>
> I expect the following line to give me a compilation error:
>
>
> https://github.com/clebertsuconic/clebert-shade-issue/blob/main/artemis-commons/src/main/java/org/apache/activemq/artemis/utils/JsonLoader.java#L38
>
>
> And that shading isolation will only work (that means give me the
> compilation error) if I remove this line from the main pom (after mvn
> install on artemis-commons-json of course):
>
> https://github.com/clebertsuconic/clebert-shade-issue/blob/main/pom.xml#L34
>
>
> if I also call "mvn dependency:tree", you will see the dependency
> listed and not giving the isolation I expected:
>
>
> https://gist.github.com/clebertsuconic/cb025175470a0f429582d3f656b48377#file-gistfile1-txt-L5-L6
>
>
>
>
>
> So, in other words, shading does not work internally. The pom
> replacement will not work through the internal dependency list.
>
> It seems like a bug to me, unless someone finds an explanation?
>
> and how I would be able to achieve this? would anybody know?
>
> On Wed, Nov 17, 2021 at 8:39 AM Matt Benson  wrote:
> >
> >
> https://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createDependencyReducedPom
> >
> > On Tue, Nov 16, 2021, 10:47 PM Clebert Suconic <
> clebert.suco...@gmail.com>
> > wrote:
> >
> > > I’m not sure how you mean.
> > >
> > > Just get the Pom at the source level or am intermediate build process ?
> > >
> > >
> > > How to do that ?
> > >
> > > On Tue, Nov 16, 2021 at 11:34 PM Matt Benson 
> wrote:
> > >
> > > > You probably want to rewrite the pom, using the property provided by
> the
> > > > shade goal for this purpose.
> > > >
> > > > Matt
> > > >
> > > > On Tue, Nov 16, 2021, 7:48 PM Clebert Suconic <
> clebert.suco...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > I am trying to create a component within ActiveMQ Artemis that
> would
> > > > > shade johnzon and javax.json.
> > > > >
> > > > > That component should then be used by other components within
> Artemis.
> > > > >
> > > > > I'm doing that because some users want to use javax.json and others
> > > > > want to use jakarta.json on their runtimes. Since we only use json
> > > > > internally I am trying to shade our own usage and not relay on
> either
> > > > > one of these package names.
> > > > >
> > > > >
> > > > > However I'm getting crazy on this. I can't make shade to hide the
> > > > > dependency. mvn dependency:tree still shows the libraries. and
> shade
> > > > > will not work if I make them provided.e.
> > > > >
> > > > >
> > > > > What is the right way to shade within my own project?
> > > > >
> > > > >
> > > > > I have the project available on my own github fork here:
> > > > >
> > > > >
> https://github.com/clebertsuconic/activemq-artemis/tree/commons-json
> > > > >
> > > > > (type this if you can download my branch:
> > > > >
> > > > > git clone https://github.com/clebertsuconic/activemq-artemis.git
> > > > > cd activemq-artemis
> > > > > git clone commons-json
> > > > > mvn install -DskipTests=true
> > > > > mvn dependency:tree
> > > > >
> > > > > and here is what gets interesting.
> > > > > if I go to artemis-selector (a package that relied on
> > > > > artemis-commons-json) and type mvn dependency:tree on that package,
> > > > > the dependency does not show up.
> > > > >
> > > > >
> > > > > The issue is only when building the whole project...
> > > > >
> > > > >
> > > > > and I have played with quite a few options! )
> > > > >
> > > > >
> > > > >
> > > > > Any help would be appreciated ! :)
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> -
> > > 

Re: Local repository accessed by multiple Maven instances - how is it solved ?

[Francois Marot]

Michael, I do not agree. As a user and maintainer of the build chain in my
company, I think Maven would really benefit from an out of the box
solution. I think any user is susceptible to the bug by launching multiple
Maven instances at the same time on his computer. Bugs then encountered are
really a bad sign sent to users, and requiring each dev to setup a database
(even simple) is cumbersome.

Regards

François

Le lun. 4 oct. 2021 à 22:13, Michael Osipov  a écrit :

> Tamás,
>
> Redis is so easy to install and get going in 5 minutes that I would
> rather see your energy go into areas which need more attention.
>
> M
>
> Am 2021-10-04 um 22:02 schrieb Tamás Cservenák:
> > Hi Bernd,
> >
> > nothing is wrong with advisory file locking, as long as you don't store
> > local repo on NFS ;)
> > Will re-add file locking once I get there, as in my opinion it is the
> most
> > "lightweight" MP (multi process) solution on a single host.
> > Redis and Hazelcast are more for "farms", where several hosts with many
> > processes (and each with many threads) is bashing local repo (that MAY be
> > on NFS as well).
> >
> > Thanks
> > Tamas
> >
> > On Mon, Oct 4, 2021 at 9:37 PM Bernd Eckenfels 
> > wrote:
> >
> >> What’s the problem with adivisory locking, as long as Maven honors the
> >> advice it is the same as it’s a redis lock? (But much less footprint).
> In
> >> fact on the same machine it should even work without locking as Long as
> you
> >> use pidfiles?
> >>
> >> Gruss
> >> Bernd
> >>
> >>
> >
>
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
> For additional commands, e-mail: users-h...@maven.apache.org
>
>

Re: Local repository accessed by multiple Maven instances - how is it solved ?

[Francois Marot]

"Salut" Michael,

thanks for the detailled answer.
Regarding my Jenkins 'hack' you can totally forget about it: it just
ensures no more than one Maven execution uses a given repo at once by
locating the local repositories in a folder reflecting the executor (which
are kinda Jenkins "threads").
I plan not to use your work right now mostly because it would involve
setting up a Redis Database and my 'hack' still works (despite using a lot
of disk space and being a bit slow).

If I understand correctly after Tams' message, your work is mostly geared
toward running multiple Maven build in parallel and on multiple hosts, all
sharing one big "local" repository through the network. Am I correct ? In
my case, I have multiple Jenkins workers but never thought about making
them collaborate on a shared "local" repo. I'll have to think about it !

Thanks !

François



Le lun. 4 oct. 2021 à 21:22, Michael Osipov  a écrit :

> Am 2021-10-04 um 14:31 schrieb Francois Marot:
> > Hello all,
> >
> > I would like clarifications on MNG-2802[*] that seems to be solved. If I
> > understand correctly it solves the problem where multiple simultaneous
> > Maven executions shared the same local repository.
> > I have been keeping for years a bit of code in my Jenkinsfiles ensuring
> the
> > local repos were accessed only by a single jenkins executor at the same
> > time, so it seems like I can get rid of this doesn't it ?
> > I'd like your input on this because reproducing the bug is difficult so
> I'd
> > like to be sure before simplifying my Jenkinsfile.
> >
> > Moreover, does anyone know how this problem is solved technically ? Using
> > files lock ? Can anybody explain ?
> >
> > Thanks you and thanks the Maven team for keeping up the good work at such
> > pace !
>
> Salut François,
>
> I don't know what you exactly fiddled in your Jenkinsfile, but there is
> no way make it right just with Jenkinsfile foo when you want access
> *one* local repo with *multiple* processes you *must* use an external
> coordinator process.
> Forget also about file locks, they are all advisory, at least on
> POSIX-like filesystems. There is no way, or a very hard way to make them
> right. We tried and removed them.
>
> Now, let me give you bit of history: I have integrated/implemented a
> fully working multi-process Redisson-based synchronization lock factory
> for Maven Resolver 1.6.x which ships with Maven 3.8.x. (It has already
> been battle tested for several months by a user on a busy CI server).
> Tamás took it to the next level in Resolver 1.7.x and generalized it to
> a named lock approach where the Redisson Lock Factory plugs in nicely
> with other lock implementations. Almost nine months of work and it works
> very good now.
> Now here is the problem: Maven Resolver 1.7.x comes only with Maven 4.x.
> I have created a branch which reconciles best of breed: Maven 3.8.x and
> Maven Resolver 1.7.x. It just works (also verified by this user). I also
> ran more than a thousand builds with 8 parallel processes and 4 threads.
> It will easily handle thousands of locks in parralel with an one digit
> percent overhead.
>
> I don't know for what documentation you are waiting for, but everything
> is there, you can start *now* to leverage it: [2], [3]. With Resolver
> 2.0.0 and Maven 4.0.0 it will likely be even easier to integrate.
>
> Hazelcast will also work, but Tamás is the special here.
>
> Again: You *must* use an external process to synchronize access. No
> excuses.
>
> Michael
>
> [1] https://github.com/apache/maven/tree/maven-3.8.x-resolver-1.7.x
> [2]
> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html
> [3]
>
> https://maven.apache.org/resolver/maven-resolver-named-locks-redisson/index.html
>
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
> For additional commands, e-mail: users-h...@maven.apache.org
>
>

Re: Local repository accessed by multiple Maven instances - how is it solved ?

[Francois Marot]

Wooohaa, thanks Thomas.
It's a whole new world being revealed to me: I wasn't aware of the huge
work happening on artifact resolvers.
At the same time, it seems not to be available out of the box (needs a
Redis server) so I'll keep my Jenkinsfile hacks for now and will wait
for proper documentation and how-to.
Thanks for the pointers !


*François Marot*



Le lun. 4 oct. 2021 à 15:10, Thomas Broyer  a écrit :

> From JIRA links, this is apparently fixed by
> https://issues.apache.org/jira/browse/MRESOLVER-131, which means you need
> to deploy a Redis server.
> There apparently also is an Hazelcast-based version:
> https://maven.apache.org/resolver/maven-resolver-named-locks/index.html
>
> On Mon, Oct 4, 2021 at 2:32 PM Francois Marot 
> wrote:
>
> > Hello all,
> >
> > I would like clarifications on MNG-2802[*] that seems to be solved. If I
> > understand correctly it solves the problem where multiple simultaneous
> > Maven executions shared the same local repository.
> > I have been keeping for years a bit of code in my Jenkinsfiles ensuring
> the
> > local repos were accessed only by a single jenkins executor at the same
> > time, so it seems like I can get rid of this doesn't it ?
> > I'd like your input on this because reproducing the bug is difficult so
> I'd
> > like to be sure before simplifying my Jenkinsfile.
> >
> > Moreover, does anyone know how this problem is solved technically ? Using
> > files lock ? Can anybody explain ?
> >
> > Thanks you and thanks the Maven team for keeping up the good work at such
> > pace !
> >
> > François MAROT
> >
> >
> > * https://issues.apache.org/jira/browse/MNG-2802
> >
>
>
> --
> Thomas Broyer
> /tɔ.ma.bʁwa.je/ <http://xn--nna.ma.xn--bwa-xxb.je/> <
> http://xn--nna.ma.xn--bwa-xxb.je/>
>

Local repository accessed by multiple Maven instances - how is it solved ?

[Francois Marot]

Hello all,

I would like clarifications on MNG-2802[*] that seems to be solved. If I
understand correctly it solves the problem where multiple simultaneous
Maven executions shared the same local repository.
I have been keeping for years a bit of code in my Jenkinsfiles ensuring the
local repos were accessed only by a single jenkins executor at the same
time, so it seems like I can get rid of this doesn't it ?
I'd like your input on this because reproducing the bug is difficult so I'd
like to be sure before simplifying my Jenkinsfile.

Moreover, does anyone know how this problem is solved technically ? Using
files lock ? Can anybody explain ?

Thanks you and thanks the Maven team for keeping up the good work at such
pace !

François MAROT


* https://issues.apache.org/jira/browse/MNG-2802

Re: Maven tries to download twice, first one without auth

[Francois Marot]

You're right, my bad...
I use:
true

because 'false' is the default. Which makes me think Maven should change
the default to true because we explicitly know the servers listed in
settings.xml and trust them.


Le lun. 30 août 2021 à 18:00, Bernd Eckenfels  a
écrit :

> Hello,
>
> Hm… I thought preemptive=false is the default - but anyway, good that it
> works now for you.
>
> Gruß
> Bernd
>
> --
> https://Bernd.eckenfels.net
> ____
> From: Francois Marot 
> Sent: Monday, August 30, 2021 4:43:33 PM
> To: Maven Users List 
> Subject: Re: Maven tries to download twice, first one without auth
>
> Hello Bernd and Slawomir,
> I want to thank you both for the advice. Using the preemptive auth works
> like a charm.
> In the end, the analysis showed that a network component was seeing all the
> "401 Unauthorized" errors and was randomly deciding
> to stop the requests. It must have been mistakenly associated to a network
> attack or something.
>
> The setting to use is this one:
>
> 
> 
> false
> 
> 
>
> I would advise anyone in a corporate network to configure its settings.xml
> with this setting to access his corporate repository.
> I even believe it should be the default setting for predefined servers
> located in settings.xml because Maven does not try to communicate with
> random servers on the internet by itself (in which case disabling
> preemptive auth may be worth it).
> Should I create an issue ?
>
>
> Le ven. 20 août 2021 à 23:12, Bernd Eckenfels  a
> écrit :
>
> > Hello,
> >
> > This is, for some reason intentional, the client is supposed to only sent
> > auth when challenged. Since this is not done in a smart way it can slow
> > transfers down.
> >
> > There used to be ways to configure it pre-emptively, but not sure if the
> > doc is still valid:
> >
> http://maven.apache.org/guides/mini/guide-http-settings.html#Example:_Using_Preemptive_Authentication
> >
> > Greetings
> > Bernd
> >
> >
> > --
> > http://bernd.eckenfels.net<http://bernd.eckenfels.networks>
> > 
> > Von: Francois Marot 
> > Gesendet: Friday, August 20, 2021 4:12:33 PM
> > An: Maven Users List 
> > Betreff: Maven tries to download twice, first one without auth
> >
> > Hello Maven users,
> >
> > I face a strange problem that I'd like to describe. I'm setting up a new
> > artifact repository and this is JFrog's Artifactory.
> > I have build freeze problems (which I still no have solved) so during my
> > investigations I found this in Artifactory's logs:
> >
> > - - - - - - - - - - - - - - - - - - - - - - - - -
> >
> >
> 2021-08-19T16:07:55.729Z|339817580d4decc9|192.168.38.137|non_authenticated_user|GET|/maven-libs-release/org/apache/velocity/velocity/1.5/velocity-1.5.pom|
> > *401*|-1|0|0|Apache-Maven/3.8.2 (Java 11.0.8; Linux
> > 3.10.0-1160.25.1.el7.x86_64)
> >
> >
> >
> 2021-08-19T16:07:55.734Z|123110ba8a2e7eb9|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/org/apache/velocity/velocity/1.5/velocity-1.5.pom|200|-1|7714|2|Apache-Maven/3.8.2
> > (Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
> >
> >
> 2021-08-19T16:07:55.743Z|f0d9c80461c879fd|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/org/apache/velocity/velocity/1.5/velocity-1.5.pom.sha1|200|-1|40|3|Apache-Maven/3.8.2
> > (Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
> >
> >
> >
> 2021-08-19T16:07:55.752Z|6500587859c8a99c|192.168.38.137|non_authenticated_user|GET|/maven-libs-release/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.pom|
> > *401*|-1|0|0|Apache-Maven/3.8.2 (Java 11.0.8; Linux
> > 3.10.0-1160.25.1.el7.x86_64)
> >
> >
> >
> 2021-08-19T16:07:55.758Z|32c027e38c5f6b49|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.pom|200|-1|12403|3|Apache-Maven/3.8.2
> > (Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
> >
> >
> 2021-08-19T16:07:55.766Z|1dce46d14ae35ed3|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.pom.sha1|200|-1|40|2|Apache-Maven/3.8.2
> > (Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
> >
> >
> >
> 2021-08-19T16:07:55.774Z|a37b50bee467027a|192.168.38.137|non_authenticated_user|GET|/maven-libs-release/org/apache/commons/commons-parent/39/commons-parent-39.pom|
> > *401*|-1|0|1|Apache-Maven/3.8.2 (Java 11.0.8; Linux
> > 3.10.0-1160.25.1.el7.x86_64)
> >
> >
> >
> 2

Re: Maven tries to download twice, first one without auth

[Francois Marot]

Hello Bernd and Slawomir,
I want to thank you both for the advice. Using the preemptive auth works
like a charm.
In the end, the analysis showed that a network component was seeing all the
"401 Unauthorized" errors and was randomly deciding
to stop the requests. It must have been mistakenly associated to a network
attack or something.

The setting to use is this one:



false



I would advise anyone in a corporate network to configure its settings.xml
with this setting to access his corporate repository.
I even believe it should be the default setting for predefined servers
located in settings.xml because Maven does not try to communicate with
random servers on the internet by itself (in which case disabling
preemptive auth may be worth it).
Should I create an issue ?


Le ven. 20 août 2021 à 23:12, Bernd Eckenfels  a
écrit :

> Hello,
>
> This is, for some reason intentional, the client is supposed to only sent
> auth when challenged. Since this is not done in a smart way it can slow
> transfers down.
>
> There used to be ways to configure it pre-emptively, but not sure if the
> doc is still valid:
> http://maven.apache.org/guides/mini/guide-http-settings.html#Example:_Using_Preemptive_Authentication
>
> Greetings
> Bernd
>
>
> --
> http://bernd.eckenfels.net<http://bernd.eckenfels.networks>
> 
> Von: Francois Marot 
> Gesendet: Friday, August 20, 2021 4:12:33 PM
> An: Maven Users List 
> Betreff: Maven tries to download twice, first one without auth
>
> Hello Maven users,
>
> I face a strange problem that I'd like to describe. I'm setting up a new
> artifact repository and this is JFrog's Artifactory.
> I have build freeze problems (which I still no have solved) so during my
> investigations I found this in Artifactory's logs:
>
> - - - - - - - - - - - - - - - - - - - - - - - - -
>
> 2021-08-19T16:07:55.729Z|339817580d4decc9|192.168.38.137|non_authenticated_user|GET|/maven-libs-release/org/apache/velocity/velocity/1.5/velocity-1.5.pom|
> *401*|-1|0|0|Apache-Maven/3.8.2 (Java 11.0.8; Linux
> 3.10.0-1160.25.1.el7.x86_64)
>
>
> 2021-08-19T16:07:55.734Z|123110ba8a2e7eb9|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/org/apache/velocity/velocity/1.5/velocity-1.5.pom|200|-1|7714|2|Apache-Maven/3.8.2
> (Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
>
> 2021-08-19T16:07:55.743Z|f0d9c80461c879fd|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/org/apache/velocity/velocity/1.5/velocity-1.5.pom.sha1|200|-1|40|3|Apache-Maven/3.8.2
> (Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
>
>
> 2021-08-19T16:07:55.752Z|6500587859c8a99c|192.168.38.137|non_authenticated_user|GET|/maven-libs-release/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.pom|
> *401*|-1|0|0|Apache-Maven/3.8.2 (Java 11.0.8; Linux
> 3.10.0-1160.25.1.el7.x86_64)
>
>
> 2021-08-19T16:07:55.758Z|32c027e38c5f6b49|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.pom|200|-1|12403|3|Apache-Maven/3.8.2
> (Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
>
> 2021-08-19T16:07:55.766Z|1dce46d14ae35ed3|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.pom.sha1|200|-1|40|2|Apache-Maven/3.8.2
> (Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
>
>
> 2021-08-19T16:07:55.774Z|a37b50bee467027a|192.168.38.137|non_authenticated_user|GET|/maven-libs-release/org/apache/commons/commons-parent/39/commons-parent-39.pom|
> *401*|-1|0|1|Apache-Maven/3.8.2 (Java 11.0.8; Linux
> 3.10.0-1160.25.1.el7.x86_64)
>
>
> 2021-08-19T16:07:55.779Z|8c4ca838bc43d3b2|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/org/apache/commons/commons-parent/39/commons-parent-39.pom|200|-1|61975|2|Apache-Maven/3.8.2
> (Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
>
> 2021-08-19T16:07:55.793Z|9d2ce78abf97f47e|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/org/apache/commons/commons-parent/39/commons-parent-39.pom.sha1|200|-1|40|3|Apache-Maven/3.8.2
> (Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
>
>
> 2021-08-19T16:07:55.801Z|8d5e469c8d929120|192.168.38.137|non_authenticated_user|GET|/maven-libs-release/org/apache/apache/16/apache-16.pom|
> *401*|-1|0|0|Apache-Maven/3.8.2 (Java 11.0.8; Linux
> 3.10.0-1160.25.1.el7.x86_64)
>
>
> 2021-08-19T16:07:55.807Z|2e1b208f4afeae52|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/org/apache/apache/16/apache-16.pom|200|-1|15507|2|Apache-Maven/3.8.2
> (Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
>
> 2021-08-19T16:07:55.815Z|b634f0399cad3c56|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/org/apache/apache/16/apache-16.pom.sha1|200|-1|40|2|Apache-Ma

Maven tries to download twice, first one without auth

[Francois Marot]

Hello Maven users,

I face a strange problem that I'd like to describe. I'm setting up a new
artifact repository and this is JFrog's Artifactory.
I have build freeze problems (which I still no have solved) so during my
investigations I found this in Artifactory's logs:

- - - - - - - - - - - - - - - - - - - - - - - - -
2021-08-19T16:07:55.729Z|339817580d4decc9|192.168.38.137|non_authenticated_user|GET|/maven-libs-release/org/apache/velocity/velocity/1.5/velocity-1.5.pom|
*401*|-1|0|0|Apache-Maven/3.8.2 (Java 11.0.8; Linux
3.10.0-1160.25.1.el7.x86_64)

2021-08-19T16:07:55.734Z|123110ba8a2e7eb9|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/org/apache/velocity/velocity/1.5/velocity-1.5.pom|200|-1|7714|2|Apache-Maven/3.8.2
(Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
2021-08-19T16:07:55.743Z|f0d9c80461c879fd|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/org/apache/velocity/velocity/1.5/velocity-1.5.pom.sha1|200|-1|40|3|Apache-Maven/3.8.2
(Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)

2021-08-19T16:07:55.752Z|6500587859c8a99c|192.168.38.137|non_authenticated_user|GET|/maven-libs-release/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.pom|
*401*|-1|0|0|Apache-Maven/3.8.2 (Java 11.0.8; Linux
3.10.0-1160.25.1.el7.x86_64)

2021-08-19T16:07:55.758Z|32c027e38c5f6b49|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.pom|200|-1|12403|3|Apache-Maven/3.8.2
(Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
2021-08-19T16:07:55.766Z|1dce46d14ae35ed3|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.pom.sha1|200|-1|40|2|Apache-Maven/3.8.2
(Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)

2021-08-19T16:07:55.774Z|a37b50bee467027a|192.168.38.137|non_authenticated_user|GET|/maven-libs-release/org/apache/commons/commons-parent/39/commons-parent-39.pom|
*401*|-1|0|1|Apache-Maven/3.8.2 (Java 11.0.8; Linux
3.10.0-1160.25.1.el7.x86_64)

2021-08-19T16:07:55.779Z|8c4ca838bc43d3b2|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/org/apache/commons/commons-parent/39/commons-parent-39.pom|200|-1|61975|2|Apache-Maven/3.8.2
(Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
2021-08-19T16:07:55.793Z|9d2ce78abf97f47e|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/org/apache/commons/commons-parent/39/commons-parent-39.pom.sha1|200|-1|40|3|Apache-Maven/3.8.2
(Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)

2021-08-19T16:07:55.801Z|8d5e469c8d929120|192.168.38.137|non_authenticated_user|GET|/maven-libs-release/org/apache/apache/16/apache-16.pom|
*401*|-1|0|0|Apache-Maven/3.8.2 (Java 11.0.8; Linux
3.10.0-1160.25.1.el7.x86_64)

2021-08-19T16:07:55.807Z|2e1b208f4afeae52|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/org/apache/apache/16/apache-16.pom|200|-1|15507|2|Apache-Maven/3.8.2
(Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)
2021-08-19T16:07:55.815Z|b634f0399cad3c56|192.168.38.137|token:ci-fake-user|GET|/maven-libs-release/org/apache/apache/16/apache-16.pom.sha1|200|-1|40|2|Apache-Maven/3.8.2
(Java 11.0.8; Linux 3.10.0-1160.25.1.el7.x86_64)

2021-08-19T16:07:55.825Z|befe40b044d34700|192.168.38.137|non_authenticated_user|GET|/maven-libs-release/commons-lang/commons-lang/2.1/commons-lang-2.1.pom|
*401*|-1|0|1|Apache-Maven/3.8.2 (Java 11.0.8; Linux
3.10.0-1160.25.1.el7.x86_64)
- - - - - - - - - - - - - - - - - - - - - - - - -

as you can see, for each pom artifact, it seems like Maven does a first
request not authenticated (hence the "|401|" in the logs) and then another
one authenticated. This second one succeeds.
I corrected this problem by adding a httpHeader property (see below) with
the same value as the "password" field (that was previously the only place
where I wrote the password) and now I have correct logs with only one
request and no 401 anymore.

- - - - - - - - - - - - - - - - - - - - - - - - -


  ci-fake-user
  XXX
  releases
 

  
Authorization
Bearer XXX
  

  


  ci-fake-user
  XXX
  snapshots
 

  
Authorization
Bearer XXX
  

  

  
- - - - - - - - - - - - - - - - - - - - - - - - -

I tested with Maven  3.6.3, 3.8.1 and 3.8.2
Do you have an explanation to my strange problem ?

Regards

François

PS: it seems Nabble forums are closed. I used to access the Maven mailing
list through Nabble (I really don't like interracting in a forum-like place
with email, I prefer good old web pages). So goodby Nabble, I'll miss you.

Re: dependency:tree without building the project (why does dependency:tree needs project artifacts installed?)

[Francois Marot]

Hello,

maybe the workaround indicated here will help you:
http://maven.40175.n5.nabble.com/Finding-SNAPSHOT-dependencies-without-compiling-first-td5998636.html#a5998639
But I'm not confident on the rasons why it works like this...


*- - - - -François Marot*


On Mon, 6 Jul 2020 at 23:16, Anton Vodonosov  wrote:

> Hello,
>
> Can anyone explain why "mvn dependency:tree" fails if the project
> artifacts are absent in ~/.m2?
>
> Also, if I place an empty, dummy file in place of the artifact in ~/.m2
> dependency:tree starts working.
> This means the artifact content is not needed for that operation.
>
> Can this be worked around somehow, to allow producing project dependency
> without building the project?
>
> Best regards,
> - Anton
>
> -
> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
> For additional commands, e-mail: users-h...@maven.apache.org
>
>

Re: Maven Artifacts not consumable after Introducing ci-friendly approach - revision tag

[Francois Marot]

beware of a caveat when using flatten and shade plugins:
https://github.com/mojohaus/flatten-maven-plugin/issues/100 &
https://stackoverflow.com/questions/52552329/use-maven-flatten-plugin-and-maven-shade-plugin-at-the-same-time

Basically you have to make sure that flatten is plugged AFTER shade in the
Maven lifecycle.
I recently faced the problem ;) :
https://twitter.com/FrancoisMarot/status/1263028183047299072


*- - - - -François Marot06 50 91 96 38*



On Tue, 16 Jun 2020 at 07:41, Hanish Bansal 
wrote:

> Still facing the same issue.
>
> Child project1 has shade plugin dependency in pom.xml.
>
> *Steps to reproduce the error:*
> 1. Compile external project
> https://github.com/hanishbansal/external-common ,
> this is some external project in which we can not change anything.
> 2. Our project "test-parent" has dependency of "external-common" project.
> Now checkout code from https://github.com/hanishbansal/test-parent adn try
> to compile it. Module "child1" is not deployed properly in local maven
> repo.
> Below is pom snippet of child1 module where revision tag is not replaced.
>  
> test-parent
> com.sample
> ${revision}
>   
>   4.0.0
>   child1
>
> On Mon, Jun 15, 2020 at 10:37 AM Hanish Bansal <
> hanish.bansal.agar...@gmail.com> wrote:
>
> > Hi All,
> >
> > Thanks for your response.
> >
> > I have replicated the same issue as mentioned in below steps:
> > 1. Compile external project
> > https://github.com/hanishbansal/external-common , this is some external
> > project in which we can not change anything.
> > 2. Our project "test-parent" has dependency of "external-common" project.
> > Now checkout code from https://github.com/hanishbansal/test-parent adn
> > try to compile it. Module "child1" is not deployed properly in local
> maven
> > repo.
> > Below is pom snippet of child1 module where revision tag is not replaced.
> >  
> > test-parent
> > com.sample
> > ${revision}
> >   
> >   4.0.0
> >   child1
> >
> >
> >
> >
> >
> > On Sat, Jun 13, 2020 at 8:18 PM Tomo Suzuki 
> > wrote:
> >
> >> Hi Hanish,
> >>
> >> I see child1 in my Maven local repository cannot resolve the $version to
> >> refer to its parent, because $version is defined at the parent (the
> error
> >> message below).
> >>
> >> How about specifying the concrete version when specifying parent? While
> >> you
> >> might not like that the version value will appear multiple places, you
> can
> >> update them by "mvn version:set"
> >> https://www.mojohaus.org/versions-maven-plugin/set-mojo.html
> >>
> >>
> >> [ERROR] Failed to execute goal on project child2: Could not resolve
> >> dependencies for project com.sample:child2:jar:1.1.0-SNAPSHOT: Failed to
> >> collect dependencies at com.sample:child1:jar:1.1.0-SNAPSHOT: Failed to
> >> read artifact descriptor for com.sample:child1:jar:1.1.0-SNAPSHOT: Could
> >> not find artifact com.sample:test-parent:pom:${revision} in central (
> >> https://repo.maven.apache.org/maven2) -> [Help 1]
> >> [ERROR]
> >> [ERROR] To see the full stack trace of the errors, re-run Maven with the
> >> -e
> >> switch.
> >> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> >> [ERROR]
> >> [ERROR] For more information about the errors and possible solutions,
> >> please read the following articles:
> >> [ERROR] [Help 1]
> >>
> >>
> http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
> >>
> >> /tmp/test-parent/child2 $ cat
> >>
> >>
> ~/.m2/repository/com/sample/child1/1.1.0-SNAPSHOT/child1-1.1.0-SNAPSHOT.pom
> >> http://maven.apache.org/POM/4.0.0; xmlns:xsi="
> >> http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="
> >> http://maven.apache.org/POM/4.0.0
> >> http://maven.apache.org/xsd/maven-4.0.0.xsd;>
> >>   4.0.0
> >>   
> >> com.sample
> >> test-parent
> >> ${revision}
> >>   
> >>   child1
> >> %
> >>
> >> On Sat, Jun 13, 2020 at 8:41 AM Francois Marot <
> francois.ma...@gmail.com>
> >> wrote:
> >>
> >> > Hello,
> >> > I can confirm that you sample project works as intended once you add
> the
> >> > missing configuration for the flatten plugin. Without it

Re: Maven Artifacts not consumable after Introducing ci-friendly approach - revision tag

[Francois Marot]

Hello,
I can confirm that you sample project works as intended once you add the
missing configuration for the flatten plugin. Without it the pom.xml
installed in your local repo are not correct because of the placeholders in
the .

I just copy pasted the  content as explained here:
http://maven.apache.org/maven-ci-friendly.html#install-deploy

Hope you will soon make it work



*- - - - -François Marot06 50 91 96 38*



On Sat, 13 Jun 2020 at 05:39, Hanish Bansal 
wrote:

> Hi Team,
>
> I have pushed a sample minimal project in git at the following link
> https://github.com/hanishbansal/test-parent
>
> Project Structure:
>   test-parent
>- child1
>- child2
>
> child2 project has dependency of child1 project.
>
> Steps to Reproduce the error:
> 1. Compile test-parent project with goal "clean install". It will compile
> and install all child projects.
> 2. Now compile child2 project, it would give compilation error due to
> dependency of child1 project.
>
>
>
> On Sat, Jun 13, 2020 at 12:58 AM Tomo Suzuki 
> wrote:
>
> > I suspect typo in child module name. Do you want to share minimum
> > reproducible project?
> >
> > On Fri, Jun 12, 2020 at 07:17 Hanish Bansal <
> > hanish.bansal.agar...@gmail.com>
> > wrote:
> >
> > > Yes, that has been already tried. I have installed child modules
> > separately
> > > but if there is any child module (e.g. child2) has dependency of other
> > > child module child1 then I am not able to compile child2 project.
> > >
> > > On Fri, Jun 12, 2020 at 3:39 PM Tomo Suzuki  >
> > > wrote:
> > >
> > > > (Assuming yy-utils is the one of the child project)
> > > > Run ”mvn install” in yy-utils module first.
> > > >
> > > > On Fri, Jun 12, 2020 at 01:56 Hanish Bansal <
> > > > hanish.bansal.agar...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > We have maven Multi Module Project, where we have recently
> > implemented
> > > > > version control process as mentioned in below documentation but
> after
> > > > > introducing it we are not able to compile child projects
> > independently
> > > > > because of dependency failure.
> > > > > http://maven.apache.org/maven-ci-friendly.html#multi-module-setup
> > > > >
> > > > > We have also used flatten-maven-plugin but artifacts deployed in
> our
> > > > > repository via this setup are not consumable by Maven anymore.
> > > > >
> > > > >  
> > > > >   
> > > > > 
> > > > >   org.codehaus.mojo
> > > > >   flatten-maven-plugin
> > > > >   1.1.0
> > > > >   
> > > > > true
> > > > > resolveCiFriendliesOnly
> > > > >   
> > > > >   
> > > > > 
> > > > >   flatten
> > > > >   process-resources
> > > > >   
> > > > > flatten
> > > > >   
> > > > > 
> > > > > 
> > > > >   flatten.clean
> > > > >   clean
> > > > >   
> > > > > clean
> > > > >   
> > > > > 
> > > > >   
> > > > > 
> > > > >   
> > > > >   
> > > > >
> > > > > While trying to get artifacts, maven build failed with below error:
> > > > >
> > > > > Failed to read artifact descriptor for
> > > > > org..xxx.x:yy-utils:jar:0.0.1: Could not transfer artifact
> > > > >
> > > > > Can you please help here?
> > > > >
> > > > > --
> > > > >
> > > > > *Thanks & Regards,Hanish Bansal*
> > > > >
> > > > --
> > > > Regards,
> > > > Tomo
> > > >
> > >
> > >
> > > --
> > >
> > > *Thanks & Regards,Hanish Bansal*
> > > Lead Software Engineer, Big Data
> > > (M) : +91-995-339-9925
> > > LinkedIn: www.linkedin.com/in/hanishbansal/
> > >
> > --
> > Regards,
> > Tomo
> >
>
>
> --
>
> *Thanks & Regards,Hanish Bansal*
> Lead Software Engineer, Big Data
> (M) : +91-995-339-9925
> LinkedIn: www.linkedin.com/in/hanishbansal/
>

Finding SNAPSHOT dependencies without compiling first

[Francois MAROT]

Hello all,

I have a simple requirement: I would like to determine very fast if my
multi-module project has any SNAPSHOT dependency.

First try was with the /mvn dependency:tree / command and grepping for the
word SNAPSHOT. But the dependency plugin seems to use the repository
dependencies, so it is not able to resolve inter-module dependencies without
first having to INSTALL the dependencies.

A workaround is to use:
/mvn compile dependency:tree -Dmaven.main.skip=true/
It seems this command initializes enough of the build plan so that Maven is
able to resolve internal (inter-modules) dependencies. The
"/maven.main.skip/" is used to optimize the speed by skipping the actual
compilation. In my case I had to also add a /-DskipSourceGeneration=true/
because a source generation step was relying on the source being compiled.

I also tried with the enforcer plugin which now allows rules to be passed on
the command line. But the result is the same, the build plan seems not to be
build hence dependency resolution is done through the repository and fails
if all jars are not INSTALLED first. As previously the workaround is to
force a fake "compile" step:

/mvn clean compile
org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M3:enforce
-Drules=requireReleaseDeps -Dmaven.main.skip=true
-DskipSourceGeneration=true/

So the question is: is there really no way to have the full dependency graph
without all those hacks ?



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Bug in Surefire 3.0.0-M4

[Francois MAROT]

Here is a self contained test in a single pom with no other files that
demonstrates the problem:
https://gist.github.com/fmarot/3bc638212e52ff1e0c55843c7fb6bbaa

The problem being that pre-existing env variable are not redefined even if
explicitly set in surefire.



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Bug in Surefire 3.0.0-M4

[Francois MAROT]

Hello all,
just want to notify of a bug we found today when switching our surefire
version from 2.x.y to the latest 3.0.0-M4 (
https://blogs.apache.org/maven/entry/apache-maven-surefire-version-3 ). It
creates problems with environment variables which seem to be incorrectly
set. As a consequence, my PATH is not set correctly and my DLL which needs
to load another DLL (and uses the PATH for this) cannot.
We switched back to 3.0.0-M3 where the bug was not present.
For now I haven't created a Short, Self Contained, Correct example, but I
could create it to help track the bug if needed. I just wanted to make sure
this was not a known issue and already reported.

Regards
François



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Augment pom dependencies information with custom ones

[Francois MAROT]

Hello again (I'm active there today ;) ),

this time I'd like to tell you about a feature I would like, and maybe some
of you will have something to advise me.
I have to generate a report listing all my dependencies (and I have tooo
many !) and for each one I have to provide a little text telling why I chose
this library.
The best way I could imagine would be to have an optional < rationale /> tag
in the < dependency /> block in order to document this directly in the pom.
But to my knowledge this is not possible (maybe for the next version of the
pom format ?!). Then a simple plugin could generate directly a report.

For the moment, I have used the license-maven-plugin which lists all the
dependencies in an html file that I have manually edited to add the info.
But it is not future proof and I will have to do it again next time ! Is
there any plugin that would take as input both your dependencies and a file
containing the choice rationale (for each dependency GAV) and would generate
an html listing dependencies and their rationale ?

Have a good day !
François



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Ensure dependency convergence restricted to a given artifactId

[Francois MAROT]

Hello all,

First of all thanks thanks and thanks again to the Maven devs, you rock !
Then a question:
is there a way to limit the dependency convergence check (in Maven enforcer
plugin -
https://maven.apache.org/enforcer/enforcer-rules/dependencyConvergence.html
) to a given artifactId (or a list of artifactId or GA[V]) ?

I ask this because I have a project spanning multiple git repos (the
"original sin") and I have to ensure convergence on a specific dependency.
But historically we have many convergence problems and it would take a lot
more effort to solve them all.

To my knowledge, the plugin is kind of all-or-nothing ? Either all converge,
or it fails. Is there a way to configure it more fine-grained or any
alternative solution ?

Regards

François MAROT



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Same snapshot deploy number for entire build - possible

[Francois Marot]

I'm sorry to insist but nevertheless I insist ;). I may have misunderstood
the problem but as I understand it, the whole problem can be sumed up by:
" the fact that the repository adds a number at the end of the deployed
files is a problem because not all artifacts deployed from the same reactor
may not get the same suffix/number if some previous build failed half way".
I still do not see how it is a problem as it is only a problem if one wants
to get each files manually from the filesystem. In this case he does not
know the *right* number for each artifact.
But if the QA want to download all the artifacts belonging to the latest
build, all he has to do is create a pom.xml referencing all those artifact,
use a variable as the version number and use the dependency plugin (
https://maven.apache.org/plugins/maven-dependency-plugin/copy-dependencies-mojo.html
) to retrieve the artifacts locally.

This way, what you see as a problem is hust an internal implementation
detail of Maven/repository that you can circumvent easily.

Again, I may have misunderstood so please excuse me if I'm talking nonsense
but I thought it could help.

Regards







*- - - - -François Marot06 50 91 96 38*



On Thu, 12 Sep 2019 at 04:07, Dan Tran  wrote:

> Hello Maven Users and Development Team
>
> Currently, artifact deployed as snapshot at Maven repository has the
> following format
>
>  artifactId-version-timestamp-NN
>
> where NN auto-incremented at each maven module and the number varies
>
> Is there a way to use same snapshot NN for the entire multi-module maven
> build?
>
> If I have to implement a solution, would it be as an extension or I have to
> tinker with maven-deploy-plugin?
>
> Very appreciated any advice
>
> -D
>

Re: Same snapshot deploy number for entire build - possible

[Francois Marot]

OK, I get the "problem" but in fact I do not think this is a problem.
How those files (with the number suffix) are named, whether you deploy (to
a remote repo) or install (to a local folder), is just an implementation
detail. You should never end up accessing those files by their name.
Let me explain: if you want to get all the 3 latest artifacts, whatever the
file name and the timestamp version, you can just create a pom having those
3 artifacts declared as dependencies and use maven dependency plugin (
https://maven.apache.org/plugins/maven-dependency-plugin/plugin-info.html )
to copy those artifacts in a directory. Use the "-U" flag to make sure to
get the latest version. This way, maven will care about the file names, but
you will not.

I hope I can help





On Fri, 13 Sep 2019 at 22:42, Tamás Cservenák  wrote:

> Howdy,
>
> so what Dan is asking for is I think the following thing:
> On multi module snapshot deploy, the last bit of snapshot timestamped
> version is "buildNo".
>
> Consider following scenario: you have a 3 module reactor build:
> i) you deploy first time: the buildNo is -1
> ii) you deploy second time, and 2nd module UT fails: result in repo is 1st
> module has -2, while the rest, as module2 failed, is not deployed
> iii) you deploy third time: the build No is -3 for 1st module, while it is
> -2 for the rest.
>
> All sounds great ("as it should be"), but buildNo has "fall apart". 3rd
> deployment will have snapshots that will have their version like
> 1.0-mmdd.hhmmss-3 while the rest version is 1.0-mmdd.hhmmss-2.
>
> Determining buildNo is possible only from maven metadata.xml, that may be
> overkill. Nothing logically "groups" the two version above, even if they
> were deployed at same time, by the same build.
>
> Deploy at end may help here, yes, but personally, I dislike the fact how
> buildNo is calculated (md get, buildNo++ jar put,, modified md put).
>
> Also, this topic seems to me, like somewhat related to the overall issue of
> https://issues.apache.org/jira/browse/MNG-6581
>
> Thanks,
> T
>
>
>
> On Fri, Sep 13, 2019 at 8:32 PM Francois Marot 
> wrote:
>
> >  can you tell us a little bit more about your problem please ?
> > You say "Currently, artifact deployed as snapshot at Maven repository has
> > the following format: artifactId-version-timestamp-NN ". Do youmean the
> > filename in the repo ? If so, why is it a problem for you ?
> > Share more with us and I think we'll suggest solution more adapted for
> your
> > use case  ;)
> >
> > Regards,
> > Francois
> >
> >
> >
> > On Thu, 12 Sep 2019 at 04:07, Dan Tran  wrote:
> >
> > > Hello Maven Users and Development Team
> > >
> > > Currently, artifact deployed as snapshot at Maven repository has the
> > > following format
> > >
> > >  artifactId-version-timestamp-NN
> > >
> > > where NN auto-incremented at each maven module and the number varies
> > >
> > > Is there a way to use same snapshot NN for the entire multi-module
> maven
> > > build?
> > >
> > > If I have to implement a solution, would it be as an extension or I
> have
> > to
> > > tinker with maven-deploy-plugin?
> > >
> > > Very appreciated any advice
> > >
> > > -D
> > >
> >
>

Re: Same snapshot deploy number for entire build - possible

[Francois Marot]

 can you tell us a little bit more about your problem please ?
You say "Currently, artifact deployed as snapshot at Maven repository has
the following format: artifactId-version-timestamp-NN ". Do youmean the
filename in the repo ? If so, why is it a problem for you ?
Share more with us and I think we'll suggest solution more adapted for your
use case  ;)

Regards,
Francois



On Thu, 12 Sep 2019 at 04:07, Dan Tran  wrote:

> Hello Maven Users and Development Team
>
> Currently, artifact deployed as snapshot at Maven repository has the
> following format
>
>  artifactId-version-timestamp-NN
>
> where NN auto-incremented at each maven module and the number varies
>
> Is there a way to use same snapshot NN for the entire multi-module maven
> build?
>
> If I have to implement a solution, would it be as an extension or I have to
> tinker with maven-deploy-plugin?
>
> Very appreciated any advice
>
> -D
>

Re: Company-wide pom and dependency management best practices

[Francois Marot]

 I remember talking with Herve Boutemy at Devoxx FR and he told me about an
upgrade of the help:effective-pom (
https://maven.apache.org/plugins/maven-help-plugin/effective-pom-mojo.html )
that would not only display the effective pom but also for each line the
source pom where this value comes from.
You can force the use of the latest help plugin with this line:
mvn org.apache.maven.plugins:maven-help-plugin:3.2.0:effective-pom
-Dverbose=true
The verbose parameter is mandatory to have the origin info as comment for
each line.
Hope it helps !


*- - - - -François Marot*



On Tue, 6 Aug 2019 at 16:50, Nick Stolwijk  wrote:

> Thanks Tomo,
>
> > I use "mvn dependency:tree -Dverbose=true" to get the detail of tree. It
> shows
> unselected nodes.
>
> The only problem with the verbose route is that it can show wrong results
> on Maven 3[1][2]. I was hoping for a non-deprecated solution.
>
> [1]
>
> https://maven.apache.org/plugins/maven-dependency-plugin/tree-mojo.html#verbose
> [2] http://maven.apache.org/shared/maven-dependency-tree/
>
> > I would also take your approach of "a dependency management pom" (BOM,
> bill of materials) to control transitive dependencies.
>
> Yeah, it seems that would be the best option.
>
> With regards,
>
> Nick Stolwijk
>
> ~~~ Try to leave this world a little better than you found it and, when
> your turn comes to die, you can die happy in feeling that at any rate you
> have not wasted your time but have done your best ~~~
>
> Lord Baden-Powell
>
>
> On Tue, Aug 6, 2019 at 4:40 PM Tomo Suzuki 
> wrote:
>
> > Hi Nick,
> >
> > > I see where the different versions for a dependency are coming from?
> >
> > I use "mvn dependency:tree -Dverbose=true" to get the detail of tree. It
> > shows unselected nodes.
> >
> > > how can I make sure that the dependencies in the Spring Boot import
> take
> > preference over the other dependency imports
> >
> > I would also take your approach of "a dependency management pom" (BOM,
> bill
> > of materials) to control transitive dependencies.
> >
> >
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Management
> >
> >
> > On Tue, Aug 6, 2019 at 10:31 AM Nick Stolwijk 
> > wrote:
> >
> > > Hi all,
> > >
> > > We have a company-wide pom for specific applications. In our case for
> > > Spring Boot applications. In this pom some of the stuff is for the
> build
> > > (plugin management, deployment management) and some for the
> dependencies
> > > (scope import for dependencyManagement).
> > >
> > > We had a buildproblem where one of the Spring Boot dependencies had the
> > > wrong version. It turned out the version of one of the dependencies for
> > > Spring Boot was overridden by another dependencyManagement import.
> > >
> > > The first question is, how can I see where the different versions for a
> > > dependency are coming from? I know of the dependency:tree, but that
> only
> > > shows the resolved version and not all the version options.
> > >
> > > The second question is, how can I make sure that the dependencies in
> the
> > > Spring Boot import take preference over the other dependency imports.
> My
> > > idea is to break apart the Spring Boot parent we have into a build pom
> > and
> > > a dependency management pom and require that every application import
> the
> > > dependency import pom as first dependency under dependency management.
> > > Would this work or am I missing something?
> > >
> > > Is there any documentation about the best practices for inheriting
> build
> > > configuration and dependency management?
> > >
> > > With regards,
> > >
> > > Nick Stolwijk
> > >
> > > ~~~ Try to leave this world a little better than you found it and, when
> > > your turn comes to die, you can die happy in feeling that at any rate
> you
> > > have not wasted your time but have done your best ~~~
> > >
> > > Lord Baden-Powell
> > >
> >
> >
> > --
> > Regards,
> > Tomo
> >
>

Re: Maven local repo in a common global directory for multiple parallel execution

[Francois MAROT]

Hi,

if you run multiple jobs concurrently but all are agnostic to one another (I
mean they only depend on Maven Central's non SNAPSHOT dependencies) and if
disk space is a problem then you can try to have a common local repo.

But if you want to play safe, you should use a per Jenkins executor repo (an
executor only runs one job at a time, and jobs should be able to share their
repo sequentially). You can read more here:
http://maven.40175.n5.nabble.com/Not-able-to-read-jars-in-repo-intermittently-tp5928990p5928998.html

Regards



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Concurrency issue while running Maven on Jenkins host

[Francois MAROT]

Sorry, seems my previous message missed the copy pasted content ! Here it is:

   pipeline {
agent { node { label paramsMap.agentParam } }// the build will
only run on nodes (ie slaves) indicated 

environment {
// To prevent simultaneous job working in the same Maven local
repo, we use per-executor local repo (more details here:
http://maven.40175.n5.nabble.com/Not-able-to-read-jars-in-repo-intermittently-td5928990.html#a5928997
)
MAVEN_CMD = "mvn -U --batch-mode  
${paramsMap.activateParallelBuild ? '-T 1C' : ''} " +
"-Dmaven.repo.local=${isUnix() ?
'$COMPANY_JENKINS_HOME.m2/repository/executor_$EXECUTOR_NUMBER' :
'%COMPANY_JENKINS_HOME%.m2\\repository\\executor_%EXECUTOR_NUMBER%'} " +
"-s ${isUnix() ? '$MAVEN_SETTINGS_XML' :
'%MAVEN_SETTINGS_XML%'} "
JAVA_TOOL_OPTIONS = "${env.JAVA_TOOL_OPTIONS == null ? '' :
env.JAVA_TOOL_OPTIONS} -Djava.io.tmpdir=${env.WORKSPACE}/target/"
}
...



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Concurrency issue while running Maven on Jenkins host

[Francois MAROT]

Hello, I suffered from similar symptoms until I adjusted the build to have
per-Jenkins-executor MAven cache. So in the end, my Maven configuration in
my Jenkins pipeline looks like this:



As you can see, I do not use any Jenkins-Maven plugin and Maven is
configured by hand. Hope it may help



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Usage of dependency-reduced-pom.xml from maven-shade-plugin in a multi-module/reactor project

[Francois MAROT]

Hi Nikki,

Regarding your question, I'm not sure of what is a "reactor of reactor"...
by 2 separate "projects" I mean 2 different build. Hence 2 different
"reactor".
As I understand it, the reactor is the sum of all Maven modules
participating in the build. Usually in the case of a multi-modules Maven
project, you start the build from the top level module, and all your modules
are part of the same rector. Part of the same build.
I personally like to have one git repository per multi-module Maven project
so we have:

1 git repo --> containing one multi-modules Maven project ⊃ composed of many
modules == one build == one reactor

What is sure is that for the shade plugin to work, you have to make Maven
builds depending on the shaded artifact find this artifact stored in a repo
(whether local or remote) and not reference it as a module inside the
reactor (ie part of the same build).

I hope it helps !



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Usage of dependency-reduced-pom.xml from maven-shade-plugin in a multi-module/reactor project

[Francois MAROT]

Hello Kyle,

to my knowledge your analysis is good, you are correct.
Sorry but I did not take the time to read your pom excerpt but I have been
using the shade plugin for quite some time now and concluded that shaded
artifacts must be leafs in the Maven modules tree. By "leaf" I mean no
artifact in the same Maven reactor must depend upon a shaded artifact. You
must separate the build of a shaded artifact and those depending upon it.

>From my understanding, this relates to the way Maven calculates the
dependencies: they are calculated at the beginning of the build, not at the
end or incrementally. While the shade plugin acts merely as a hack and
replaces the pom at the end of building the shaded module. It's too late  in
the process as the dependencies of artifacts depending on the shaded one
have already been calculated.

My advice would be to ensure you have 2 separated projects, not part of the
same reactor build to prevent problems or surprise for newcomers.

Regards
Francois Marot



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Applying Jenkins build number

[Francois MAROT]

The new way to deal with such situation is described here:
https://maven.apache.org/maven-ci-friendly.html


0-SNAPSHOT


1.2.${revision}

and let Jenkins run Maven like this mvn deploy
-Drevision=${env.BUILD_NUMBER} (or maybe just $BUILD_NUMBER depending on the
way you run Jenkins)



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Parent not deployed/

[Francois MAROT]

Hi,

I have the same "problem" here which is not really a problem: I deploy the
parent pom (and the parent's parent and so on) and voila. As parent are only
pom, no real package or jars, this not really a problem.
But I think the Flatten Maven Plugin
https://www.mojohaus.org/flatten-maven-plugin/ may be what you are looking
for. It flattens your pom so no parent pom is needed.



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

RE: .m2 repository migration

[Francois MAROT]

Yaron is right: you may find problems in your build by NOT moving the
.m2/repository folder. You may discover that specific files were available
only in your local repo and not anywhere else (Maven Central or your local
proxy such as Archiva/Artifactory/Nexus). And this is a good time to
discover bad practise in your build.
But the answer to the original question is YES, simply copy the directory
.m2/repository and inspect any specific setting in .m2/settings.xml.



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Using Maven 3 with non-standard repository layout (Ivy)

[Francois MAROT]

I would guess that Ivy can work a lot better with Maven repositories than
Maven can with Ivy repositories. I would investigate a one-shot scripted
migration of your Ivy repo to a new Maven repo and force existing publishers
to the Ivy repo to configure Ivy to publish to the new Maven repo.
But I can imagine that it involves politics to make existing Ivy publishers
change their publish mechanism...



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Problem with exec-maven-plugin

[Francois MAROT]

Hi Alex,

maybe you could paste here the output logs of the failing execution. I'd
like them to be sure I understand the problem.
Cheers

Francois



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Not able to read jars in repo intermittently

[Francois MAROT]

For the record, it seems like the "EXECUTOR_NUMBER" variable (
https://issues.jenkins-ci.org/browse/JENKINS-31391 ) should be available to
be able to easily specify a per-executor repo.
So something such as this should do the job:
mvn -Dmaven.repo.local=$JENKINS_HOME/.m2/$EXECUTOR_NUMBER/ ...



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Not able to read jars in repo intermittently

[Francois MAROT]

I regularly have the same problem on my Jenkins build server: either
"corrupted zip file" or "cannot find symbol" and the only explanation for me
was concurrent build. I has never thought of having Maven local repo
specific to an executor, thanks Bernd Eckenfels, that's a really good point
!
I was really worried about the *huge* disk impact on disk space per-job
repositories would have but had never thought about per-executor ! Thanks.
To tell you the truth, I was thinking about a system to have a file-system
layer to mutualize all the non-SNAPSHOT jars and another per-job layer to
store the SNAPSHOT jars... whatever, this would have been complicated and
the per-executor seems like a good trade-of between simplicity and disk
space.

Francois



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: ConcurrentContinuous Delivery Maven Builds

[Francois MAROT]

What do you mean by "pushing maven-metadata.xml which can be corrupted" ?
Are you talking about https://issues.apache.org/jira/browse/MDEPLOY-221
which is solved in Maven 3.5.2 (even 3.5.1 I think) ?



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Make custom plugin executed only once in multimodule (inherited=false? aggregator?)

[Francois MAROT]

Thanks Robert, Karl, I created a JIRA MNG-6336 here:
https://issues.apache.org/jira/browse/MNG-6336



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Make custom plugin executed only once in multimodule (inherited=false? aggregator?)

[Francois MAROT]

Thanks Robert, that makes sense. At least it explains why aggregator will not
work in my case.
Do you think my case is not so uncommon and I should open a bug, or I
misunderstood "the Maven way" and have to adapt ?

Regards



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Make custom plugin executed only once in multimodule (inherited=false? aggregator?)

[Francois MAROT]

Thanks for your response Hervé,

I've made a sample plugin here:
https://github.com/fmarot/print-dummy-maven-plugin
When included in the "" section of a parent pom of a
multimodule Maven project, the behavior is shown: when running 'mvn
validate', "dummy" is printed for for each and every child execution and in
the parent.
As I set aggregator to "true" in my Mojo, I expected it to be executed only
once.

I can provide a sample multimodule if needed.

Regards



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Make custom plugin executed only once in multimodule (inherited=false? aggregator?)

[Francois MAROT]

[Sorry, re-posting with correct formatting]

Hi all & happy new year !
I have a question regarding plugin development.

I have made a custom plugin and would like to make it's Mojo executed once
*and only once* in my whole reactor build. If I include it in my parent pom
(not in pluginManagement), then by default it is executed once at the root
and once in child-modules.

Solutions I found out:

- false used in plugin declaration in pom => it sorta
do what I want 
BUT my plugin is not executed once if I invoke the maven build directly
inside a child-module.

- aggregator = true in the @Mojo annotation => I thought it was THE solution
but I can't
seem to understand what it does: despite declaring my mojo an aggregator, it
is executed in all children.

-
http://blog.sonatype.com/2009/05/how-to-make-a-plugin-run-once-during-a-build/
=> the solution given there does not really satisfies me because it means
Maven WILL call my execute() method more than once, simply my plugin will
bypass it except at the execution root. We can do better than that can't we
?!

I really can't understand why the aggregator parameter does not work as I
expected it to.
Any idea ?

François 



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Make custom plugin executed only once in multimodule (inherited=false? aggregator?)

[Francois MAROT]

Hi all & happy new year !I have a question regarding plugin development.I
have made a custom plugin and would like to make it's Mojo executed once
*and only once* in my whole reactor build.If I include it in my parent pom
(not in pluginManagement), then by default it is executed once at the root
and once in child-modules. Solutions I found out:- false used in plugin
declaration in pom => it sorta do what I want BUT my plugin is not executed
once if I invke the maven build directly inside a child-module.- aggregator
= true in the @Mojo annotation => I thought it was THE solution but I can't
seem to understand what it does: despite declaring my mojo an aggregator, it
is executed in all children.-
http://blog.sonatype.com/2009/05/how-to-make-a-plugin-run-once-during-a-build/
=> the solution given there does not really satisfies me because it means
Maven WILL call my execute() method more than once, simply my plugin will
bypass it except at the execution root. We can do better than that can't we
?!I really can't understand why the aggregator parameter does not work as I
expected it to.Any idea ?François



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

Re: Best way to deploy a specific file and declare its dependencies

[Francois MAROT]

Thanks for the reply Olivier, but I don't think I can "attach" . Let me
explain:

Basically, I want to deploy someJar.jar and at the same time declare a
dependency:someJar.jar --> redist-cpp.zip

redist-cpp.zip is already stored in Archiva.
someJar.jar is not produced by Maven.

If I "attach" someJar.jar, then I have to attach it to a dummy empty main
artifact that would serve no purpose other than to let someJar.jar being
deployed alongside.
Moreover, I think (not sure) that the dependencies of the main artifact are
not dependencies of the attached artifacts. So I would not achieve my goal.

I can not "attach" redist-cpp.zip because itis already stored on Archiva.

I could maybe have used  mvn deploy:deploy-file
  
and use option "-DpomFile=" to point to a pom where I would define the
dependency to redist-cpp.zip.
Problem is that I would not be able to pass the version of my artifact on
the command-line (see  Maven CI Friendly stuff
  ), I would have to change
it in the pom on each deploy.





--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Best way to deploy a specific file and declare its dependencies

[Francois MAROT]

Hello all,

I have a need to explain to a third party how to deploy a jar on my
repository (Archiva). But
- this jar is not generated by Maven (it contains DLLs to be loaded through
JNA)
- I'd like the third party to be able to declare dependencies of this jar
(some zip containing some DLLs). So that by declaring a dependency on this
jar I will also get it's dependencies
- the goal is for me to make the third party totally independant and able to
update his artifact or it's dependencies without my supervision

I came with the following pom that he may use runnning:
mvn install -Drevision=1.0.0-WIN64-SNAPSHOT
-DfileToDeploy=some-jar_master.jar

Question is, is there anything simpler ? :)


http://www.w3.org/2001/XMLSchema-instance;
xmlns="http://maven.apache.org/POM/4.0.0;
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/maven-v4_0_0.xsd;>
4.0.0



com.oleamedical.io
test-breast
${revision}

jar



UTF-8
TO-OVERRIDE-ON-COMMAND-LINE
TO-OVERRIDE-ON-COMMAND-LINE






thirdparties.intel
redist-cpp
2017.5.267
natives-win
runtime
zip









maven-antrun-plugin
1.8


package






run







org.codehaus.mojo
flatten-maven-plugin
1.0.1

true



flatten
process-resources

flatten



flatten.clean
clean

clean












--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: maven plugin testing harness

[Francois MAROT]

I'm no expert at all, but just to let you know: last time I tried the test
harness plugin, I also gave up... :(
I think some advocate to use the maven-invoker-plugin to test in real
conditions with much less dependency problems. You can have a look there:
https://maven.apache.org/plugin-developers/plugin-testing.html#maven-invoker-plugin



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Access the full dependency tree in a Mojo (Maven 3)

[Francois MAROT]

Hi all,

I'm writing a plugin whose goal is to output a dependency graph *at the scm
level*. Meaning I don't really care about individual dependencies but about
the source control manager (git usually) where they come from.
I hope it will help to know in which order we must release all our internal
dependencies.
https://github.com/fmarot/scm-graph-maven-plugin

I've been inspired by the Maven Dependency tree plugin but it seems that
even this plugin does not handle full dependency tree anymore:
*
https://github.com/apache/maven-plugins/blob/b09c595a7cc6352aef55d2738d0382d0b8ef8c01/maven-dependency-plugin/src/main/java/org/apache/maven/plugins/dependency/tree/TreeMojo.java#L233
* https://issues.apache.org/jira/browse/MDEP-374
* https://issues.apache.org/jira/browse/MDEP-339
* ...
It used to use Maven2 specific dependency resolution which was now
inconsistent with Maven 3.

Does anybody know how I can get the full dependency graph in my own Mojo ?
It would make my plugin much more usefull !

Best regards
Francois



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Windows and Linux build for same GAV different classifier

[Francois MAROT]

Hi Kevin,

I think you are facing more or less the same problem I faced before (
http://maven.40175.n5.nabble.com/mvn-deploy-deploy-file-with-same-groupId-amp-artifactId-but-different-classifiers-td5838931.html#a5838934
).
So to sum up:
- if you want to add 2 native artifacts differing by the classifier, the
ideal way is to deploy both at the same time as the main artifact.
Otherwise, if deployed separately, it seems the behavior may vary depending
on the artifact repository.
- if deploying at the same time is difficult (so was my case), then you can
declare different artifactId and voila. It's not as clean and nice but it
may be easier to manage.

I also take the opportunity to advertise my "native" plugin which may help
you: https://github.com/fmarot/nativedependencies-maven



--
Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

New plugin to handle native dependencies: nativedependencies-maven-plugin

[Francois MAROT]

Hello everyone,

I wanted to advertise a "new" Maven plugin that may help you deal with
native dependencies (.so, .dll, or whatever, as you may find other uses).
It is forked from an existing but inactive plugin (
https://code.google.com/archive/p/mavennatives/ ) where I added a few
features.
I just published v 1.0.0 on Maven Central.
I hope to add a lot more documentation on the project's Github wiki in the
future and document the best way to deal with native dependencies,
especially regarding tests and multi-module.
Any help is welcome, feel free to improve / suggest / open feature requests.

All the information on the Github site:
https://github.com/fmarot/nativedependencies-maven

Hope it may be of interest for some of you.
Regards

François



--
View this message in context: 
http://maven.40175.n5.nabble.com/New-plugin-to-handle-native-dependencies-nativedependencies-maven-plugin-tp5860832.html
Sent from the Maven - Users mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: How to perform a deploy only

[Francois MAROT]

Hello Pavan,

it's hard to tell exactly but in Eclipse (Eclipse JEE at least) if you
declare a tomcat instance in the "server" window, and import your maven
modules with m2e, you should be able to deploy to Tomcat without problems.
No more needs for command-line clean/install/deploy (as long as your project
does not do very dirty stuffs).
Check the "m2e-wtp" keyword on Google as it's the plugin doing the bridge
between Maven in Eclipse and the server deployment stuffs...



--
View this message in context: 
http://maven.40175.n5.nabble.com/How-to-perform-a-deploy-only-tp118100p5851470.html
Sent from the Maven - Users mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: Convention for shipped artefacts?

[Francois MAROT]

I have kinda the sae project: 50 or so modules, 3 'leaf' artifacts (I coined
the term 'leaf' to designate the artifacts that will actually be shipping
outside.
For me those 3 artifacts share the fact they are each an obfuscated
über-jar, and each composed of a combination of the other artifacts. As they
are obfuscated, I created an "obfuscated-parent" module (of type 'pom') to
contain the 3 leaf artifacts.
So people now know that they find the modules visibles to end-users in the
'obfuscated-parent' directory/module.

It's the best I came up with.



--
View this message in context: 
http://maven.40175.n5.nabble.com/Convention-for-shipped-artefacts-tp5851343p5851467.html
Sent from the Maven - Users mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

RE: [EXTERNAL] Re: mvn deploy:deploy-file with same groupId artifactId but different classifiers

[Francois MAROT]

OK all, so I've taken the not so clean but should solve my problem faster
road... I publish the 2 zips separatly on Archiva.
But now, my main build that declare dependencies on them both (as 'runtime')
seems to randomly fail, looking for one or the other. If I publish one of
the 2, the main build starts passing again.

It's strange and I fell like it has to do something with the time. Any idea
?

my tests are always made using -U to force updates of dependencies



--
View this message in context: 
http://maven.40175.n5.nabble.com/mvn-deploy-deploy-file-with-same-groupId-artifactId-but-different-classifiers-tp5838931p5839121.html
Sent from the Maven - Users mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

RE: [EXTERNAL] Re: mvn deploy:deploy-file with same groupId artifactId but different classifiers

[Francois MAROT]

OK, so Maven only sees the most recent one uploaded because on Archiva, the
maven-metadata.xml file is updated each time with the timestamp of the
most recent one (either *-linux or *-win). But the 2 qualified artifacts do
not share the same timestamp so one of those is not found leading to build
failed.
Conclusion: I will declare 2 different artifacts for now and, when I have
time, I will create a meta jenkins task launching the build on windows and
linux simultaneously and uploading both at the same time to Archiva.



--
View this message in context: 
http://maven.40175.n5.nabble.com/mvn-deploy-deploy-file-with-same-groupId-artifactId-but-different-classifiers-tp5838931p5839126.html
Sent from the Maven - Users mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: mvn deploy:deploy-file with same groupId artifactId but different classifiers

[Francois Marot]

@Jeff, thank you I think I'll try that. I thnink it will solve my problem
@Wolf: you're totally right. That is the only solution I thought of before
Jeff's. It's the cleaner solution, but as I'm in the middle of a migration,
I will take some shortcuts to finish sooner. But at the same time I fear
the obscure behaviour. Hope I won't face it soon ;)
@Sandra: no problem, at least thanks for helping !

On a more general discussion, what do you people think ? Is Wolf's solution
the closest to The Maven Way™ or is Jeff's solution totally acceptable
(even if it's not, I'll take it for now !) ?



- - - - -


*François Marot06-50-91-96-38*

On 29 June 2015 at 11:18, Jeff MAURY jeffma...@jeffmaury.com wrote:

 I think you should have a look at the generatePom parameter (

 https://maven.apache.org/plugins/maven-deploy-plugin/deploy-file-mojo.html#generatePom
 )
 which is true by default and should be false for the second call.

 Jeff

 On Mon, Jun 29, 2015 at 11:11 AM, Francois Marot francois.ma...@gmail.com
 
 wrote:

  Thanks Sandra,
 
  but I'm not sure I understand: I already use the -Dclassifier= parameter
 in
  the 2 command I copy/pasted above.
  Do you mean setting linux/windows instead of
  natives-linux/natives-win would change something ? I think
 classifiers
  are form-free and I kind of need the prefix native- in front of it so
  that other parts of the build can handle those dependencies as native
 ones
  (for example this plugni: https://code.google.com/p/mavennatives/)
 
 
 
  - - - - -
 
 
  *François Marot06-50-91-96-38*
 
  On 29 June 2015 at 09:57, Sandra Parsick spars...@web.de wrote:
 
   Hi,
  
   you can use the property -Dclassifier. For the Linux build I would use
   -Dclassifier=linux and for the Windows build -Dclassifier=win.
  
   Best regards,
  
   Sandra
  
   Am 29.06.2015 um 09:53 schrieb Francois MAROT:
Hello all,
   
I'm in the process of converting an existing build to Maven. I have
2 Jenkins jobs building the same native library on 2 different
systems (win / lin). At the end of each one, I want to deploy them
on my Archiva repo. So I execute the following commands :
   
On a Linux computer: mvn deploy:deploy-file
-Dfile=myLib-x64-natives-linux.zip -Dclassifier=natives-linux
-DgroupId=com.myCompany  -DartifactId=myLib -Dversion=1.0.0
-DrepositoryId=archiva.internal
-Durl=http://centos-dev:8090/repository/internal/
   
On a Windows computer: mvn deploy:deploy-file
-Dfile=jre-x64-natives-win.zip -Dclassifier=natives-win
-DgroupId=com.myCompany  -DartifactId=myLib -Dversion=1.0.0
-DrepositoryId=archiva.internal
-Durl=http://centos-dev:8090/repository/internal/
   
Problem is the second command leads to a build failed because the
pom of the project in version 1.0.0 is already deployed... BUT, I
am lucky because the artifact with classifier natives-win has
already been uploaded so both artifact end up on the server
correctly. So builds depending on them can retreive them correctly
and everything is fine... ... except that I don't want to base my
build on luck and would like to know what is the correct way to
deploy 2 artifacts with everything being the same except the
qualifier.
   
How do you pepole deploy a same artifact but built by different
computers, architectures ?
   
I'm using Maven 3.3.1  Apache Archiva 2.1.1
   
   
   
-- View this message in context:
   
  
 
 http://maven.40175.n5.nabble.com/mvn-deploy-deploy-file-with-same-groupId-artifactId-but-different-classifiers-tp5838931.html
   
   
   Sent from the Maven - Users mailing list archive at Nabble.com.
   
-
   
   
   To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org
   
   
  
   -
   To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
   For additional commands, e-mail: users-h...@maven.apache.org
  
  
 



 --
 Jeff MAURY


 Legacy code often differs from its suggested alternative by actually
 working and scaling.
  - Bjarne Stroustrup

 http://www.jeffmaury.com
 http://riadiscuss.jeffmaury.com
 http://www.twitter.com/jeffmaury

mvn deploy:deploy-file with same groupId artifactId but different classifiers

[Francois MAROT]

Hello all,

I'm in the process of converting an existing build to Maven.
I have 2 Jenkins jobs building the same native library on 2 different
systems (win / lin). At the end of each one, I want to deploy them on my
Archiva repo.
So I execute the following commands :

On a Linux computer:
mvn deploy:deploy-file -Dfile=myLib-x64-natives-linux.zip
-Dclassifier=natives-linux  -DgroupId=com.myCompany  -DartifactId=myLib
-Dversion=1.0.0 -DrepositoryId=archiva.internal 
-Durl=http://centos-dev:8090/repository/internal/

On a Windows computer:
mvn deploy:deploy-file -Dfile=jre-x64-natives-win.zip
-Dclassifier=natives-win  -DgroupId=com.myCompany  -DartifactId=myLib
-Dversion=1.0.0 -DrepositoryId=archiva.internal 
-Durl=http://centos-dev:8090/repository/internal/

Problem is the second command leads to a build failed because the pom of
the project in version 1.0.0 is already deployed...
BUT, I am lucky because the artifact with classifier natives-win has
already been uploaded so both artifact end up on the server correctly. So
builds depending on them can retreive them correctly and everything is
fine...
... except that I don't want to base my build on luck and would like to know
what is the correct way to deploy 2 artifacts with everything being the same
except the qualifier.

How do you pepole deploy a same artifact but built by different computers,
architectures ?

I'm using Maven 3.3.1  Apache Archiva 2.1.1



--
View this message in context: 
http://maven.40175.n5.nabble.com/mvn-deploy-deploy-file-with-same-groupId-artifactId-but-different-classifiers-tp5838931.html
Sent from the Maven - Users mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: mvn deploy:deploy-file with same groupId artifactId but different classifiers

[Francois Marot]

Thanks Sandra,

but I'm not sure I understand: I already use the -Dclassifier= parameter in
the 2 command I copy/pasted above.
Do you mean setting linux/windows instead of
natives-linux/natives-win would change something ? I think classifiers
are form-free and I kind of need the prefix native- in front of it so
that other parts of the build can handle those dependencies as native ones
(for example this plugni: https://code.google.com/p/mavennatives/)



- - - - -


*François Marot06-50-91-96-38*

On 29 June 2015 at 09:57, Sandra Parsick spars...@web.de wrote:

 Hi,

 you can use the property -Dclassifier. For the Linux build I would use
 -Dclassifier=linux and for the Windows build -Dclassifier=win.

 Best regards,

 Sandra

 Am 29.06.2015 um 09:53 schrieb Francois MAROT:
  Hello all,
 
  I'm in the process of converting an existing build to Maven. I have
  2 Jenkins jobs building the same native library on 2 different
  systems (win / lin). At the end of each one, I want to deploy them
  on my Archiva repo. So I execute the following commands :
 
  On a Linux computer: mvn deploy:deploy-file
  -Dfile=myLib-x64-natives-linux.zip -Dclassifier=natives-linux
  -DgroupId=com.myCompany  -DartifactId=myLib -Dversion=1.0.0
  -DrepositoryId=archiva.internal
  -Durl=http://centos-dev:8090/repository/internal/
 
  On a Windows computer: mvn deploy:deploy-file
  -Dfile=jre-x64-natives-win.zip -Dclassifier=natives-win
  -DgroupId=com.myCompany  -DartifactId=myLib -Dversion=1.0.0
  -DrepositoryId=archiva.internal
  -Durl=http://centos-dev:8090/repository/internal/
 
  Problem is the second command leads to a build failed because the
  pom of the project in version 1.0.0 is already deployed... BUT, I
  am lucky because the artifact with classifier natives-win has
  already been uploaded so both artifact end up on the server
  correctly. So builds depending on them can retreive them correctly
  and everything is fine... ... except that I don't want to base my
  build on luck and would like to know what is the correct way to
  deploy 2 artifacts with everything being the same except the
  qualifier.
 
  How do you pepole deploy a same artifact but built by different
  computers, architectures ?
 
  I'm using Maven 3.3.1  Apache Archiva 2.1.1
 
 
 
  -- View this message in context:
 
 http://maven.40175.n5.nabble.com/mvn-deploy-deploy-file-with-same-groupId-artifactId-but-different-classifiers-tp5838931.html
 
 
 Sent from the Maven - Users mailing list archive at Nabble.com.
 
  -
 
 
 To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
  For additional commands, e-mail: users-h...@maven.apache.org
 
 

 -
 To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
 For additional commands, e-mail: users-h...@maven.apache.org

Re: What does my application actually require?

[Francois MAROT]

Hi Dan,

I suggest you simply run this command:

mvn dependency:copy-dependencies

it will copy all dependencies of your project into ./target/dependencies (or
something similar, I don't exactly remember).
Then, after running mvn package, just start your program by using the
wildcard to list all jars previously copied by maven to the dependencies
folder. Something such as:

java -cp target/youJar.jar:./target/dependencies/*:. yourPackage.YourClass



--
View this message in context: 
http://maven.40175.n5.nabble.com/What-does-my-application-actually-require-tp5828705p5828855.html
Sent from the Maven - Users mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

Re: How to avoid unzip of large test data on each mvn test

[Francois MAROT]

Thanks Graham, this makes a lot of sense and I totally agree this would be
the best. But as I also have to convince colleagues and to make compromises
I hope maybe someone knew a magical plugin ;)

and no, the data I work with is large and a whole and people do not want to
work with a small subset :/



--
View this message in context: 
http://maven.40175.n5.nabble.com/How-to-avoid-unzip-of-large-test-data-on-each-mvn-test-tp5801435p5801741.html
Sent from the Maven - Users mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org

How to avoid unzip of large test data on each mvn test

[Francois MAROT]

Hi all,

I'm in the process of switching to Maven for a large ANT project and have a
question regarding my test data.
I'd like those data to be versionned and stored in the Maven repo. But
currently those data are huge (let's say a 1-2 gigabytes once zipped). So I
imagine that if stored in the repo, each time I will run mvn test Maven
will have to unzip this big zip. It will take a lot of time...
Is there a best practise about it ?
Is there a way to store the data already unzipped somewhere and have mvn
test unzip the archive only if the version has changed ?
Any idea ?

François

PS: currently, the existing test code rely on the data being manually
unzipped once and for all at a hardcoded location on the disk :/ . Those
test data are currently not versionned (with all the related problems).



--
View this message in context: 
http://maven.40175.n5.nabble.com/How-to-avoid-unzip-of-large-test-data-on-each-mvn-test-tp5801435.html
Sent from the Maven - Users mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org