Re: New Committers - community

[Hervé BOUTEMY]

yes, on our 100 Git repos [1], 90% should be easy to run on any CI with 
minimum configuration and resources

then there is Maven core and complex plugins (like Surefire) which are not the 
same story

if someone wants to test other toolings to show some useful feature we don't 
get in our current ASF Jenkins configuration [2], just start with some 
examples of the base 90% and make a demo of something that would be useful to 
add to our configuration

Regards,

Hervé

[1] https://maven.apache.org/scm.html#Maven_Sources_Overview

[2] https://github.com/apache/maven-jenkins-lib

Le mercredi 2 janvier 2019, 15:38:12 CET Enrico Olivelli a écrit :
> Il giorno mer 2 gen 2019 alle ore 15:34 Vladimir Sitnikov
> 
>  ha scritto:
> > Enrico> For instance maven-surefire
> > integration tests take 2 hours.
> > In general I see Travis (free edition) does not offer many resources.
> > 
> > Each Travis job is limited by 50 minutes, however one can split tests into
> > multiple jobs, so it will run fine.
> 
> I guess it won't be easy.
> Maven integration testing is very complex, it's Maven launching Maven
> itself. We will need to improve maven-invoker plugin, I don't know if there
> is support for something like JUnit categories.
> We can run only "unit tests" and this will be a minimal "feedback" for
> the contributor.
> For simpler plugins, like "checkstyle", I think Travis would be a good
> choice, at least I would give it a try
> 
> Enrico
> 
> > Vladimir
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org





-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

[ANN] Apache Maven Assembly Plugin 3.1.1 Released

[Enrico Olivelli]

The Apache Maven team is pleased to announce the release of the Apache
Maven Assembly Plugin, version 3.1.1

The Assembly Plugin for Maven is primarily intended to allow users to
aggregate the project output along with its dependencies, modules,
site documentation, and other files into a single distributable
archive.

https://maven.apache.org/plugins/maven-assembly-plugin/

You should specify the version in your project's plugin configuration:


  org.apache.maven.plugins
  maven-assembly-plugin
  3.1.1


You can download the appropriate sources etc. from the download page:

https://maven.apache.org/plugins/maven-assembly-plugin/download.cgi

Release Notes - Maven Assembly Plugin - Version 3.1.1

** Bug
* [MASSEMBLY-675] - Maven Assembly packaging wildcard-excluded dependencies
* [MASSEMBLY-762] - Assembly plugin doesn't exclude transitive
dependencies when excluded by wildcards in dependencies section
* [MASSEMBLY-799] - Exclusion on wildcard, then the assembly would
still package to include the excluded libraries
* [MASSEMBLY-861] - exclusion * also packaged
* [MASSEMBLY-873] - Maven-Assembly-Plugin freezes when building
jar-with-dependencies of project depending on
org.bouncycastle:bcprov-jdk15on:1.58
* [MASSEMBLY-893] - Typo in FAQ

** Task
* [MASSEMBLY-885] - remove unused unpack code
* [MASSEMBLY-898] - upgrade to plexus-io 3.1.1
* [MASSEMBLY-899] - Make deprecated and non used of parameter
useJvmChmod parameter (plugin is now 1.7)
* [MASSEMBLY-901] - Fix trivial javadocs HTML errors
* [MASSEMBLY-902] - Some Integration tests fails if launched with
an very large UID

** Dependency upgrade
* [MASSEMBLY-876] - Upgrade parent to 31
* [MASSEMBLY-882] - Upgrade mave-surefire/failsafe-plugin 2.21.0
* [MASSEMBLY-884] - Upgrade plexus-archiver to 3.6.0
* [MASSEMBLY-890] - Upgrade plexus-interpolation to 1.25
* [MASSEMBLY-892] - Upgrade maven-plugins parent to version 33
* [MASSEMBLY-900] - Upgrade plexus-archiver to 4.0.0


Enjoy,

-The Apache Maven team

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: parallelise not overlapping tasks

[Romain Manni-Bucau]

Yes, in theory. In practise they are partially about sequencing cause maven
has nothing else handy (the merge in a multi module project is a headache
for most users i met). Also you cant not respect phases today (regarding my
impl).

Last point is that id the remark was on my last note, this is trivial to
fix by aliasing mojo executions - explicitly or not. The phases are plain
useless since a user will call a mojo, either with its dependencies to
replace phases (mvn +jar for example) or without (mvn jar).

Overall point is that phases now limit maven likely more it helps so for
maven 4 it is something to consider. Short term and for maven 3 i think a
compromise can be found to boost maven builds without requiring the hack to
create 5 modules where 1 is needed - which is current workaround.

Le mer. 2 janv. 2019 18:02, Stephen Connolly <
stephen.alan.conno...@gmail.com> a écrit :

> phases are about the user calling out how far they want to go, not about
> sequencing... at least once we get a propper execution graph
>
> On Thu, 6 Dec 2018 at 13:54, Romain Manni-Bucau 
> wrote:
>
> > Currently maven can't but I expect a way to do it, either in the next xsd
> > as originally proposed or, why not, with a naming convention in the id of
> > the execution (my-exec#after#other-exec or
> something
> > like that if we want it before maven 4)
> >
> > The nice thing is that once done it makes phases pretty much useless (it
> is
> > just about making implicit these dependencies) and it makes the whole
> build
> > parallelizable and not just modules which will often find some bottleneck
> > modules in projects building a distribution.
> >
> > Romain Manni-Bucau
> > @rmannibucau  |  Blog
> >  | Old Blog
> >  | Github <
> > https://github.com/rmannibucau> |
> > LinkedIn  | Book
> > <
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > >
> >
> >
> > Le jeu. 6 déc. 2018 à 14:44, Mickael Istria  a
> écrit :
> >
> > > Hi,
> > >
> > > > any way we move that topic forward beginning of next year?
> > >
> > > I guess providing patches would be the best way ;)
> > >
> > > I'm coming late in this discussion and I'm a newcomer on that list, so
> I
> > > could miss context. This could relate to an effort we are doing in
> > Eclipse
> > > IDE (and Eclipse m2e) to run module builds in parallel. One question I
> > have
> > > is how do you know 2 tasks aren't conflicting? We didn't figure out a
> > safe
> > > way to know that in m2e, maybe I missed something?
> > >
> > > Cheers,
> > >
> >
>

Re: parallelise not overlapping tasks

[Stephen Connolly]

phases are about the user calling out how far they want to go, not about
sequencing... at least once we get a propper execution graph

On Thu, 6 Dec 2018 at 13:54, Romain Manni-Bucau 
wrote:

> Currently maven can't but I expect a way to do it, either in the next xsd
> as originally proposed or, why not, with a naming convention in the id of
> the execution (my-exec#after#other-exec or something
> like that if we want it before maven 4)
>
> The nice thing is that once done it makes phases pretty much useless (it is
> just about making implicit these dependencies) and it makes the whole build
> parallelizable and not just modules which will often find some bottleneck
> modules in projects building a distribution.
>
> Romain Manni-Bucau
> @rmannibucau  |  Blog
>  | Old Blog
>  | Github <
> https://github.com/rmannibucau> |
> LinkedIn  | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le jeu. 6 déc. 2018 à 14:44, Mickael Istria  a écrit :
>
> > Hi,
> >
> > > any way we move that topic forward beginning of next year?
> >
> > I guess providing patches would be the best way ;)
> >
> > I'm coming late in this discussion and I'm a newcomer on that list, so I
> > could miss context. This could relate to an effort we are doing in
> Eclipse
> > IDE (and Eclipse m2e) to run module builds in parallel. One question I
> have
> > is how do you know 2 tasks aren't conflicting? We didn't figure out a
> safe
> > way to know that in m2e, maybe I missed something?
> >
> > Cheers,
> >
>

Re: parallelise not overlapping tasks

[Romain Manni-Bucau]

Hi guys,

I did a quick PoC on a small project. It parallelize the mojo per phase
with a config taken from project properties - not sexy but enables to start
"seeing" something.
You can find it at https://github.com/rmannibucau/parallel-mojo-extension

The main issue it enounters is the fact it can't reuse the ExecutorService
(or its CompletionService wrapper) from the MultiThreadedBuilder - since it
is not done in maven core - and the config which is pretty heavy at the
moment.

However the interesting part is that it is doable, the two main challenges
being:

1. ensure the plugins are safe (not sure threadSafe can be reused, I would
assume that yes but some structure can be not thread safe so not fully sure
and it would be ok while mojo are not parallelize in a single module)
2. find a nice configuration - maybe in terms of dependency between mojo?

The 2 is interesting cause it asks the question of phases which would be
less useful in that context.

Romain Manni-Bucau
@rmannibucau  |  Blog
 | Old Blog
 | Github  |
LinkedIn  | Book



Le sam. 8 déc. 2018 à 13:14, Romain Manni-Bucau  a
écrit :

> Using the scm is not enough or it is for single module projects.
>
> You have to have a graph of dependencies (inputs/outputs) and save each
> task state in target to have incremental support
>
> But please note incremental build is != parallel build at mojo level.
>
> This last one is easy to do and a quick win IMHO
>
> Le sam. 8 déc. 2018 00:50, Tibor Digana  a écrit :
>
>> In my projects, the most plugins use single execution.
>> External projects also have this kind of principle.
>> Thus we should have a look in those possibilities where the most plugins
>> can gain the performance.
>> Usually the compiler and tests take long.
>> I know that maven-compiler-plugin:3.8.1 will be incremental which is good
>> of course but we should perhaps continue with gaining the build
>> performance.
>> If somebody has an idea on how to develop a compiler which partially
>> compiles a module depending on SCM changes, feel free to bring it to our
>> mailing list. The same with tests where the set of tests is changed
>> depending on SCM changes.
>>
>> Cheers
>> Tibor
>>
>>
>> On Fri, Jan 19, 2018 at 2:21 PM Romain Manni-Bucau > >
>> wrote:
>>
>> > Hi guys,
>> >
>> > there is no way to parallelize not conflicting tasks for a same phase at
>> > the moment right? Any way it gets under the radar?
>> >
>> > A common example is to run all code analyzis concurrently (findbugs,
>> pmd,
>> > checkstyle, ...) at the same time without waiting for one then the other
>> > etc since all can be very long.
>> >
>> > It could be nice an fancy to define part of the reactor
>> parallelisablity in
>> > the pom, like:
>> >
>> > 
>> > 
>> > process-sources
>> > 
>> > 
>> >   ...
>> >   ...
>> >   ...
>> >   ...
>> > 
>> > 
>> >   ...
>> >   ...
>> >   ...
>> >   ...
>> > 
>> > 
>> > 
>> > 
>> >
>> > anything to enhance it?
>> >
>> > Romain Manni-Bucau
>> > @rmannibucau  |  Blog
>> >  | Old Blog
>> >  | Github <
>> > https://github.com/rmannibucau> |
>> > LinkedIn 
>> >
>>
>

Re: New Committers - community

[Enrico Olivelli]

Il giorno mer 2 gen 2019 alle ore 15:34 Vladimir Sitnikov
 ha scritto:
>
> Enrico> For instance maven-surefire
> integration tests take 2 hours.
> In general I see Travis (free edition) does not offer many resources.
>
> Each Travis job is limited by 50 minutes, however one can split tests into
> multiple jobs, so it will run fine.

I guess it won't be easy.
Maven integration testing is very complex, it's Maven launching Maven itself.
We will need to improve maven-invoker plugin, I don't know if there is
support for something like JUnit categories.
We can run only "unit tests" and this will be a minimal "feedback" for
the contributor.
For simpler plugins, like "checkstyle", I think Travis would be a good
choice, at least I would give it a try

Enrico

>
> Vladimir

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: New Committers - community

[Vladimir Sitnikov]

Enrico> For instance maven-surefire
integration tests take 2 hours.
In general I see Travis (free edition) does not offer many resources.

Each Travis job is limited by 50 minutes, however one can split tests into
multiple jobs, so it will run fine.

Vladimir

Re: New Committers - community

[Enrico Olivelli]

Il giorno mer 2 gen 2019 alle ore 15:12 Vladimir Sitnikov
 ha scritto:
>
> Enrico> Probably Travis won't be able to run our ITs, at least the free
> version
>
> Why do you think so?

Because Maven CI jobs are huge. For instance maven-surefire
integration tests take 2 hours.
In general I see Travis (free edition) does not offer many resources.

I would like to try to setup Travis for some "small" plugin and make
some test, at least testing java 11 + Linux

Enrico

>
> Vladimir

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: New Committers - community

[Vladimir Sitnikov]

Enrico> Probably Travis won't be able to run our ITs, at least the free
version

Why do you think so?

Vladimir