Re: Maven Memory Consumption

2016-06-02 Thread Jason Dillon 
colors for Linux are not exactly the same as the screen dump: yellow from the 
screen dump is bold white on Linux. This is ok for me 
Terminals can render whatever colors they want for the ANSI colors and many 
have this configurable.

To be clear my terminal env has bold color rendering as yellow (very very very 
old preference I’ve had for a long time so much I forgot it was my preference).

We can certainly make things explicitly yellow though if that is preferred over 
simply bold.

—jason

Re: Maven Memory Consumption

2016-06-02 Thread Jason Dillon 
If its on by default I would expect folks to set 
MAVEN_OPTS=-Dmaven.logging=plain instead of magically making —batch do that.

If we mutate the cli api slightly to expose more details about the cli 
configuration to the Slf4jConfiguration then regular -Dmaven.logging=plain on 
command line would probably work too.  Right now the logging configuration has 
no context of the command-line params and can only use System.properties to 
fiddle with configuration.

If ^^^ was done then could also consider adding —color={yes|no} flag, though I 
felt odd hacking that in given that this is a pluggable aspect, and if you were 
using logbook backend it would be meaningless and potentially confusing.

—jason


On June 2, 2016 at 8:37:18 PM, Manfred Moser (manf...@simpligility.com) wrote:

If we plan to switch it to on be default at a later stage we could 
automatically disable it in batch mode. And tell people to run in batch mode on 
a CI server.  

Just a thought..  

Manfred  

Jason van Zyl wrote on 2016-06-02 19:52:  

> If the output comes out decently in color in CI consoles then it’s probably  
> not an issue putting the color on by default. But I haven’t checked and  
> suggested that the color be off by default to start with.  
>  
>> On Jun 2, 2016, at 5:15 PM, Hervé BOUTEMY  wrote:  
>>  
>> I merged the PR in the slf4j-gossip branch (and added a little improvement)  
>>  
>> core ITs are ok (notice: ran without activating colors)  
>> colors for Linux are not exactly the same as the screen dump: yellow from 
>> the  
>> screen dump is bold white on Linux. This is ok for me  
>>  
>> Now, what's annoying is that:  
>> - color is not enabled by default: I had to configure MAVEN_OPTS="-  
>> Dmaven.logging=color"  
>> - when redirecting content to file, color is not disabled automatically  
>>  
>> I don't know if this is a showstopper or not  
>> I will continue to use it to see if there are unexpected side effects  
>>  
>> Regards,  
>>  
>> Hervé  
>>  
>> Le jeudi 2 juin 2016 09:21:46 Tamás Cservenák a écrit :  
>>> Olivier, if you refer to the slf4j-gossip branch, that IMHO Jason's PR  
>>> supersedes it.  
>>> Will drop that branch.  
>>>  
>>> On Thu, Jun 2, 2016 at 8:38 AM Olivier Lamy  wrote:  
 well I think this color stuff has already been done differently but never  
 accepted..  
  
 On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:  
> another feature that would be great for this release:  
> https://github.com/apache/maven/pull/81  
>  
> I still didn't have time to work on it, but I like the screenshot  
> The only thing that I'd like to check is: is tty detection working? ie  
  
 does  
  
> color automatically disappear if there is no tty?  
>  
> Regards,  
>  
> Hervé  
>  
> Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :  
>> +1  
>> this is something that was often seen: this is great that it is fixed!  
>>  
>> For example, the last time I published Maven core site, the build  
  
 simply  
  
>> failed because of PermgenSpace: now it is working like a charm...  
>>  
>> This release will be a must!  
>>  
>> Regards,  
>>  
>> Hervé  
>>  
>> Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :  
>>> Hi Manfred,  
>>>  
>>> On 6/1/16 12:24 AM, Manfred Moser wrote:  
 I can feel your excitement coming through in the emails.. ;-)  
>>>  
>>> Of course I'm excited ;-)  
>>>  
>>> ...cause it's very important...I have had heard many customers  
>>> saying  
>>> they will not upgrade to newer versions of Maven exactly based on  
  
 such  
  
>>> issue(s)...  
>>>  
>>> which is in general bad ...  
>>>  
>>>  
>>> This will break their argument ;-)...  
>>>  
 Karl Heinz Marbaise wrote on 2016-05-31 15:14:  
> Hi,  
>  
> tested without the patch (-Xmx6g) ...run time for the test  
> project  
>  
> more  
>  
> than two 2 Minutes  
>  
> running with the patch (-Xmx1g):  
>  
> Run time ca. 27 seconds...  
>  
> also worked with -Xmx768m ...ca. 30 seconds...  
>  
> so looks very good...  
>  
> Let us wait what the IT's say...  
>  
> Kind regards  
> Karl Heinz Marbaise  
>  
> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:  
>> Hi,  
>>  
>> after more investigation and an extremly good tip of  
  
 Andriy...(see  
  
>> MNG-6030) and in the end the solution:  
>>  
>> Using test project with 5000 modules just doing:  
>>  
>> mvn clean  
>>  
>> using the patch now in master  
>>

Re: Maven Memory Consumption

2016-06-02 Thread Jason Dillon 
On June 2, 2016 at 3:15:35 PM, Hervé BOUTEMY (herve.bout...@free.fr) wrote:
I merged the PR in the slf4j-gossip branch (and added a little improvement) 

core ITs are ok (notice: ran without activating colors) 
The testsuite certainly will not be very happy with color enabled as many tests 
are expecting exact strings in the output.  I had considered setting 
MAVEN_SKIP_RC env-var to avoid any local customization from polluting the 
testsuite execution.



colors for Linux are not exactly the same as the screen dump: yellow from the 
screen dump is bold white on Linux. This is ok for me 
Terminals can render whatever colors they want for the ANSI colors and many 
have this configurable.



Now, what's annoying is that: 
- color is not enabled by default: I had to configure MAVEN_OPTS="- 
Dmaven.logging=color" 
Seemed safer off by default then breaking the output for users where ansi is 
not supported and for whatever reason its not been detected as such to strip 
out.



- when redirecting content to file, color is not disabled automatically 
It may depend on how its redirected, my local testing shows that `mvn > foo’ 
disables ansi sequences, though I can not say I’ve done any exhaustive testing 
of this.

To be able to tell if System.out is an ANSI-aware terminal it may need a bit 
more love ala jline or more helpers in jansi to do more properly?

—jason

Re: Maven Memory Consumption

2016-06-02 Thread Igor Fedorenko 
Somewhat related, Jason convinced me to opensource better logging
support for multithreaded builds I implemented some time ago.

https://github.com/takari/concurrent-build-logger

-- 
Regards,
Igor

On Thu, Jun 2, 2016, at 11:37 PM, Manfred Moser wrote:
> If we plan to switch it to on be default at a later stage we could
> automatically disable it in batch mode. And tell people to run in batch
> mode on a CI server. 
> 
> Just a thought.. 
> 
> Manfred
> 
> Jason van Zyl wrote on 2016-06-02 19:52:
> 
> > If the output comes out decently in color in CI consoles then it’s probably
> > not an issue putting the color on by default. But I haven’t checked and
> > suggested that the color be off by default to start with.
> > 
> >> On Jun 2, 2016, at 5:15 PM, Hervé BOUTEMY  wrote:
> >> 
> >> I merged the PR in the slf4j-gossip branch (and added a little improvement)
> >> 
> >> core ITs are ok (notice: ran without activating colors)
> >> colors for Linux are not exactly the same as the screen dump: yellow from 
> >> the 
> >> screen dump is bold white on Linux. This is ok for me
> >> 
> >> Now, what's annoying is that:
> >> - color is not enabled by default: I had to configure MAVEN_OPTS="-
> >> Dmaven.logging=color"
> >> - when redirecting content to file, color is not disabled automatically
> >> 
> >> I don't know if this is a showstopper or not
> >> I will continue to use it to see if there are unexpected side effects
> >> 
> >> Regards,
> >> 
> >> Hervé
> >> 
> >> Le jeudi 2 juin 2016 09:21:46 Tamás Cservenák a écrit :
> >>> Olivier, if you refer to the slf4j-gossip branch, that IMHO Jason's PR
> >>> supersedes it.
> >>> Will drop that branch.
> >>> 
> >>> On Thu, Jun 2, 2016 at 8:38 AM Olivier Lamy  wrote:
>  well I think this color stuff has already been done differently but never
>  accepted..
>  
>  On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:
> > another feature that would be great for this release:
> > https://github.com/apache/maven/pull/81
> > 
> > I still didn't have time to work on it, but I like the screenshot
> > The only thing that I'd like to check is: is tty detection working? ie
>  
>  does
>  
> > color automatically disappear if there is no tty?
> > 
> > Regards,
> > 
> > Hervé
> > 
> > Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :
> >> +1
> >> this is something that was often seen: this is great that it is fixed!
> >> 
> >> For example, the last time I published Maven core site, the build
>  
>  simply
>  
> >> failed because of PermgenSpace: now it is working like a charm...
> >> 
> >> This release will be a must!
> >> 
> >> Regards,
> >> 
> >> Hervé
> >> 
> >> Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
> >>> Hi Manfred,
> >>> 
> >>> On 6/1/16 12:24 AM, Manfred Moser wrote:
>  I can feel your excitement coming through in the emails.. ;-)
> >>> 
> >>> Of course I'm excited ;-)
> >>> 
> >>> ...cause it's very important...I have had heard many customers
> >>> saying
> >>> they will not upgrade to newer versions of Maven exactly based on
>  
>  such
>  
> >>> issue(s)...
> >>> 
> >>> which is in general bad ...
> >>> 
> >>> 
> >>> This will break their argument ;-)...
> >>> 
>  Karl Heinz Marbaise wrote on 2016-05-31 15:14:
> > Hi,
> > 
> > tested without the patch (-Xmx6g) ...run time for the test
> > project
> > 
> > more
> > 
> > than two 2 Minutes
> > 
> > running with the patch (-Xmx1g):
> > 
> > Run time ca. 27 seconds...
> > 
> > also worked with -Xmx768m ...ca. 30 seconds...
> > 
> > so looks very good...
> > 
> > Let us wait what the IT's say...
> > 
> > Kind regards
> > Karl Heinz Marbaise
> > 
> > On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> >> Hi,
> >> 
> >> after more investigation and an extremly good tip of
>  
>  Andriy...(see
>  
> >> MNG-6030) and in the end the solution:
> >> 
> >> Using test project with 5000 modules just doing:
> >> 
> >> mvn clean
> >> 
> >> using the patch now in master
> >> (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> >> extremely reduces the memory footprint...
> > 
> > https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > 
> >> /M
> >> aven340-with-patch-5000.png
> >> 
> >> 
> >> This shows the result using the patch
> > 
> >> The following shows Maven 3.3.9:
> > https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > 
> >> /M

Re: Maven Memory Consumption

2016-06-02 Thread Manfred Moser 
If we plan to switch it to on be default at a later stage we could 
automatically disable it in batch mode. And tell people to run in batch mode on 
a CI server. 

Just a thought.. 

Manfred

Jason van Zyl wrote on 2016-06-02 19:52:

> If the output comes out decently in color in CI consoles then it’s probably
> not an issue putting the color on by default. But I haven’t checked and
> suggested that the color be off by default to start with.
> 
>> On Jun 2, 2016, at 5:15 PM, Hervé BOUTEMY  wrote:
>> 
>> I merged the PR in the slf4j-gossip branch (and added a little improvement)
>> 
>> core ITs are ok (notice: ran without activating colors)
>> colors for Linux are not exactly the same as the screen dump: yellow from 
>> the 
>> screen dump is bold white on Linux. This is ok for me
>> 
>> Now, what's annoying is that:
>> - color is not enabled by default: I had to configure MAVEN_OPTS="-
>> Dmaven.logging=color"
>> - when redirecting content to file, color is not disabled automatically
>> 
>> I don't know if this is a showstopper or not
>> I will continue to use it to see if there are unexpected side effects
>> 
>> Regards,
>> 
>> Hervé
>> 
>> Le jeudi 2 juin 2016 09:21:46 Tamás Cservenák a écrit :
>>> Olivier, if you refer to the slf4j-gossip branch, that IMHO Jason's PR
>>> supersedes it.
>>> Will drop that branch.
>>> 
>>> On Thu, Jun 2, 2016 at 8:38 AM Olivier Lamy  wrote:
 well I think this color stuff has already been done differently but never
 accepted..
 
 On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:
> another feature that would be great for this release:
> https://github.com/apache/maven/pull/81
> 
> I still didn't have time to work on it, but I like the screenshot
> The only thing that I'd like to check is: is tty detection working? ie
 
 does
 
> color automatically disappear if there is no tty?
> 
> Regards,
> 
> Hervé
> 
> Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :
>> +1
>> this is something that was often seen: this is great that it is fixed!
>> 
>> For example, the last time I published Maven core site, the build
 
 simply
 
>> failed because of PermgenSpace: now it is working like a charm...
>> 
>> This release will be a must!
>> 
>> Regards,
>> 
>> Hervé
>> 
>> Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
>>> Hi Manfred,
>>> 
>>> On 6/1/16 12:24 AM, Manfred Moser wrote:
 I can feel your excitement coming through in the emails.. ;-)
>>> 
>>> Of course I'm excited ;-)
>>> 
>>> ...cause it's very important...I have had heard many customers
>>> saying
>>> they will not upgrade to newer versions of Maven exactly based on
 
 such
 
>>> issue(s)...
>>> 
>>> which is in general bad ...
>>> 
>>> 
>>> This will break their argument ;-)...
>>> 
 Karl Heinz Marbaise wrote on 2016-05-31 15:14:
> Hi,
> 
> tested without the patch (-Xmx6g) ...run time for the test
> project
> 
> more
> 
> than two 2 Minutes
> 
> running with the patch (-Xmx1g):
> 
> Run time ca. 27 seconds...
> 
> also worked with -Xmx768m ...ca. 30 seconds...
> 
> so looks very good...
> 
> Let us wait what the IT's say...
> 
> Kind regards
> Karl Heinz Marbaise
> 
> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
>> Hi,
>> 
>> after more investigation and an extremly good tip of
 
 Andriy...(see
 
>> MNG-6030) and in the end the solution:
>> 
>> Using test project with 5000 modules just doing:
>> 
>> mvn clean
>> 
>> using the patch now in master
>> (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
>> extremely reduces the memory footprint...
> 
> https://github.com/khmarbaise/maven-test-project-generator/blob/master
> 
>> /M
>> aven340-with-patch-5000.png
>> 
>> 
>> This shows the result using the patch
> 
>> The following shows Maven 3.3.9:
> https://github.com/khmarbaise/maven-test-project-generator/blob/master
> 
>> /M
>> aven339-5000.png
>> 
>> 
>> Many thanks to Andriy for the support and help...
>> 
>> we will see if not IT's will fail on the change.
>> 
>> 
>> Kind regards
>> Karl Heinz Marbaise
>> 
>> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
>>> Hi,
>>> 
>>> i started a little bit more detailed analysis..
>>> 
>>> very simple via JConsole and running the different versions...
>>> 
>>> I have

Re: Integrating maven with other tooling

2016-06-02 Thread Igor Fedorenko 
On Mon, May 30, 2016, at 11:34 PM, James Roper wrote:
> 
> One question that I'm trying to solve now, when resolving dependencies,
> if
> you want to do so without actually building the project it seems that you
> would probably have to create your own WorkspaceReader implementation
> that
> will return a dummy artifact when the resolver tries to resolve the jar
> artifact, is that correct?  I've tried mavens reactor WorkspaceReader,
> but
> it only returns the pom artifact, the jar artifact remains not found.
> 

To make sure I understand the question. You have two projects, A and B,
and project A depends on B. So the question is, how to resolve
dependencies of project A. Did I get the question right?

 I think the answer depends on what you are trying to do. When m2e runs
 Maven build with "resolve workspace dependencies" enabled, for example,
 dependencies on workspace projects are always resolved to project
 target/classes directories, which works reasonably well for compiler
 and many other Maven plugins. If you just need to display list of
 project A dependency coordinates, actual file does not really matter.

... or did I completely misunderstand the question?

-- 
Regards,
Igor


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Maven Memory Consumption

2016-06-02 Thread Igor Fedorenko 
I wonder if this fix also solves "remote shared parent" memory
inefficiency described in https://issues.apache.org/jira/browse/MNG-5669

-- 
Regards,
Igor

On Tue, May 31, 2016, at 06:14 PM, Karl Heinz Marbaise wrote:
> Hi,
> 
> tested without the patch (-Xmx6g) ...run time for the test project more 
> than two 2 Minutes
> 
> running with the patch (-Xmx1g):
> 
> Run time ca. 27 seconds...
> 
> also worked with -Xmx768m ...ca. 30 seconds...
> 
> so looks very good...
> 
> Let us wait what the IT's say...
> 
> Kind regards
> Karl Heinz Marbaise
> 
> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> > Hi,
> >
> > after more investigation and an extremly good tip of Andriy...(see
> > MNG-6030) and in the end the solution:
> >
> > Using test project with 5000 modules just doing:
> >
> > mvn clean
> >
> > using the patch now in master (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> > extremely reduces the memory footprint...
> >
> > https://github.com/khmarbaise/maven-test-project-generator/blob/master/Maven340-with-patch-5000.png
> >
> >
> > This shows the result using the patch
> >
> > The following shows Maven 3.3.9:
> >
> > https://github.com/khmarbaise/maven-test-project-generator/blob/master/Maven339-5000.png
> >
> >
> > Many thanks to Andriy for the support and help...
> >
> > we will see if not IT's will fail on the change.
> >
> >
> > Kind regards
> > Karl Heinz Marbaise
> >
> > On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
> >> Hi,
> >>
> >> i started a little bit more detailed analysis..
> >>
> >> very simple via JConsole and running the different versions...
> >>
> >> I have summarized this here:
> >>
> >> https://github.com/khmarbaise/maven-test-project-generator
> >>
> >> Kind regards
> >> Karl Heinz Marbaise
> >>
> >> On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
> >>> Hi to all,
> >>>
> >>> i have a question concerning the memory consumption...
> >>>
> >>> If i run maven with the same JDK and the same reactor and build with the
> >>> same parameter and plugins...
> >>>
> >>> will the printout at the end of the build (Final Memory) something
> >>> realiable about the consumption of the JVM during the build ?...Or is it
> >>> at least a hint...or would i need to do something different (BTW:
> >>> Someone has a hint about that?) ...
> >>>
> >>>
> >>> [INFO] BUILD SUCCESS
> >>> [INFO]
> >>> 
> >>> [INFO] Total time: 6.431 s
> >>> [INFO] Finished at: 2016-04-17T17:46:58+02:00
> >>> [INFO] Final Memory: 47M/638M
> >>>
> >>> So if i ran the same build with different Maven versions so could this
> >>> give us a hint where more memory is consumed ...(to identify where and
> >>> why is a different story)...
> >>>
> >>> Kind regards
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
> 

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Maven Memory Consumption

2016-06-02 Thread Jason van Zyl 
If the output comes out decently in color in CI consoles then it’s probably not 
an issue putting the color on by default. But I haven’t checked and suggested 
that the color be off by default to start with.

> On Jun 2, 2016, at 5:15 PM, Hervé BOUTEMY  wrote:
> 
> I merged the PR in the slf4j-gossip branch (and added a little improvement)
> 
> core ITs are ok (notice: ran without activating colors)
> colors for Linux are not exactly the same as the screen dump: yellow from the 
> screen dump is bold white on Linux. This is ok for me
> 
> Now, what's annoying is that:
> - color is not enabled by default: I had to configure MAVEN_OPTS="-
> Dmaven.logging=color"
> - when redirecting content to file, color is not disabled automatically
> 
> I don't know if this is a showstopper or not
> I will continue to use it to see if there are unexpected side effects
> 
> Regards,
> 
> Hervé
> 
> Le jeudi 2 juin 2016 09:21:46 Tamás Cservenák a écrit :
>> Olivier, if you refer to the slf4j-gossip branch, that IMHO Jason's PR
>> supersedes it.
>> Will drop that branch.
>> 
>> On Thu, Jun 2, 2016 at 8:38 AM Olivier Lamy  wrote:
>>> well I think this color stuff has already been done differently but never
>>> accepted..
>>> 
>>> On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:
 another feature that would be great for this release:
 https://github.com/apache/maven/pull/81
 
 I still didn't have time to work on it, but I like the screenshot
 The only thing that I'd like to check is: is tty detection working? ie
>>> 
>>> does
>>> 
 color automatically disappear if there is no tty?
 
 Regards,
 
 Hervé
 
 Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :
> +1
> this is something that was often seen: this is great that it is fixed!
> 
> For example, the last time I published Maven core site, the build
>>> 
>>> simply
>>> 
> failed because of PermgenSpace: now it is working like a charm...
> 
> This release will be a must!
> 
> Regards,
> 
> Hervé
> 
> Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
>> Hi Manfred,
>> 
>> On 6/1/16 12:24 AM, Manfred Moser wrote:
>>> I can feel your excitement coming through in the emails.. ;-)
>> 
>> Of course I'm excited ;-)
>> 
>> ...cause it's very important...I have had heard many customers
>> saying
>> they will not upgrade to newer versions of Maven exactly based on
>>> 
>>> such
>>> 
>> issue(s)...
>> 
>> which is in general bad ...
>> 
>> 
>> This will break their argument ;-)...
>> 
>>> Karl Heinz Marbaise wrote on 2016-05-31 15:14:
 Hi,
 
 tested without the patch (-Xmx6g) ...run time for the test
 project
 
 more
 
 than two 2 Minutes
 
 running with the patch (-Xmx1g):
 
 Run time ca. 27 seconds...
 
 also worked with -Xmx768m ...ca. 30 seconds...
 
 so looks very good...
 
 Let us wait what the IT's say...
 
 Kind regards
 Karl Heinz Marbaise
 
 On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> Hi,
> 
> after more investigation and an extremly good tip of
>>> 
>>> Andriy...(see
>>> 
> MNG-6030) and in the end the solution:
> 
> Using test project with 5000 modules just doing:
> 
> mvn clean
> 
> using the patch now in master
> (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> extremely reduces the memory footprint...
 
 https://github.com/khmarbaise/maven-test-project-generator/blob/master
 
> /M
> aven340-with-patch-5000.png
> 
> 
> This shows the result using the patch
 
> The following shows Maven 3.3.9:
 https://github.com/khmarbaise/maven-test-project-generator/blob/master
 
> /M
> aven339-5000.png
> 
> 
> Many thanks to Andriy for the support and help...
> 
> we will see if not IT's will fail on the change.
> 
> 
> Kind regards
> Karl Heinz Marbaise
> 
> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
>> Hi,
>> 
>> i started a little bit more detailed analysis..
>> 
>> very simple via JConsole and running the different versions...
>> 
>> I have summarized this here:
>> 
>> https://github.com/khmarbaise/maven-test-project-generator
>> 
>> Kind regards
>> Karl Heinz Marbaise
>> 
>> On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
>>> Hi to all,
>>> 
>>> i have a question concerning the memory consumption...
>>> 
>>> If i run maven with the same JDK

Re: Maven Memory Consumption

2016-06-02 Thread Hervé BOUTEMY 
I merged the PR in the slf4j-gossip branch (and added a little improvement)

core ITs are ok (notice: ran without activating colors)
colors for Linux are not exactly the same as the screen dump: yellow from the 
screen dump is bold white on Linux. This is ok for me

Now, what's annoying is that:
- color is not enabled by default: I had to configure MAVEN_OPTS="-
Dmaven.logging=color"
- when redirecting content to file, color is not disabled automatically

I don't know if this is a showstopper or not
I will continue to use it to see if there are unexpected side effects

Regards,

Hervé

Le jeudi 2 juin 2016 09:21:46 Tamás Cservenák a écrit :
> Olivier, if you refer to the slf4j-gossip branch, that IMHO Jason's PR
> supersedes it.
> Will drop that branch.
> 
> On Thu, Jun 2, 2016 at 8:38 AM Olivier Lamy  wrote:
> > well I think this color stuff has already been done differently but never
> > accepted..
> > 
> > On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:
> > > another feature that would be great for this release:
> > > https://github.com/apache/maven/pull/81
> > > 
> > > I still didn't have time to work on it, but I like the screenshot
> > > The only thing that I'd like to check is: is tty detection working? ie
> > 
> > does
> > 
> > > color automatically disappear if there is no tty?
> > > 
> > > Regards,
> > > 
> > > Hervé
> > > 
> > > Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :
> > > > +1
> > > > this is something that was often seen: this is great that it is fixed!
> > > > 
> > > > For example, the last time I published Maven core site, the build
> > 
> > simply
> > 
> > > > failed because of PermgenSpace: now it is working like a charm...
> > > > 
> > > > This release will be a must!
> > > > 
> > > > Regards,
> > > > 
> > > > Hervé
> > > > 
> > > > Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
> > > > > Hi Manfred,
> > > > > 
> > > > > On 6/1/16 12:24 AM, Manfred Moser wrote:
> > > > > > I can feel your excitement coming through in the emails.. ;-)
> > > > > 
> > > > > Of course I'm excited ;-)
> > > > > 
> > > > > ...cause it's very important...I have had heard many customers
> > > > > saying
> > > > > they will not upgrade to newer versions of Maven exactly based on
> > 
> > such
> > 
> > > > > issue(s)...
> > > > > 
> > > > > which is in general bad ...
> > > > > 
> > > > > 
> > > > > This will break their argument ;-)...
> > > > > 
> > > > > > Karl Heinz Marbaise wrote on 2016-05-31 15:14:
> > > > > >> Hi,
> > > > > >> 
> > > > > >> tested without the patch (-Xmx6g) ...run time for the test
> > > > > >> project
> > > 
> > > more
> > > 
> > > > > >> than two 2 Minutes
> > > > > >> 
> > > > > >> running with the patch (-Xmx1g):
> > > > > >> 
> > > > > >> Run time ca. 27 seconds...
> > > > > >> 
> > > > > >> also worked with -Xmx768m ...ca. 30 seconds...
> > > > > >> 
> > > > > >> so looks very good...
> > > > > >> 
> > > > > >> Let us wait what the IT's say...
> > > > > >> 
> > > > > >> Kind regards
> > > > > >> Karl Heinz Marbaise
> > > > > >> 
> > > > > >> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> > > > > >>> Hi,
> > > > > >>> 
> > > > > >>> after more investigation and an extremly good tip of
> > 
> > Andriy...(see
> > 
> > > > > >>> MNG-6030) and in the end the solution:
> > > > > >>> 
> > > > > >>> Using test project with 5000 modules just doing:
> > > > > >>> 
> > > > > >>> mvn clean
> > > > > >>> 
> > > > > >>> using the patch now in master
> > > > > >>> (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> > > > > >>> extremely reduces the memory footprint...
> > > 
> > > https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > > 
> > > > > >>> /M
> > > > > >>> aven340-with-patch-5000.png
> > > > > >>> 
> > > > > >>> 
> > > > > >>> This shows the result using the patch
> > > 
> > > > > >>> The following shows Maven 3.3.9:
> > > https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > > 
> > > > > >>> /M
> > > > > >>> aven339-5000.png
> > > > > >>> 
> > > > > >>> 
> > > > > >>> Many thanks to Andriy for the support and help...
> > > > > >>> 
> > > > > >>> we will see if not IT's will fail on the change.
> > > > > >>> 
> > > > > >>> 
> > > > > >>> Kind regards
> > > > > >>> Karl Heinz Marbaise
> > > > > >>> 
> > > > > >>> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
> > > > >  Hi,
> > > > >  
> > > > >  i started a little bit more detailed analysis..
> > > > >  
> > > > >  very simple via JConsole and running the different versions...
> > > > >  
> > > > >  I have summarized this here:
> > > > >  
> > > > >  https://github.com/khmarbaise/maven-test-project-generator
> > > > >  
> > > > >  Kind regards
> > > > >  Karl Heinz Marbaise
> > > > >  
> > > > >  On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
> > > > > > Hi to all,
> > > > > > 
> > > > > > i have a question concerning the memory consumption...
> > > > >

[GitHub] maven issue #75: Patch for MNG-5956

2016-06-02 Thread nhojpatrick 
Github user nhojpatrick commented on the issue:

https://github.com/apache/maven/pull/75
  
I totally forgot about this defect.
@khmarbaise yep I would like to use modules multiple times, if your 
building everything then yes they are included multiple times. The build time 
improvements come when having to rebuild a portition of the multi module 
project.
@michael-o It might looks like a mess but allowing this would help reduce 
the workarounds I've seen.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

[GitHub] maven issue #75: Patch for MNG-5956

2016-06-02 Thread nhojpatrick 
Github user nhojpatrick commented on the issue:

https://github.com/apache/maven/pull/75
  
Totally forgot about this defect, as I was waiting for it to be merged in.

A git repo might might contain two deploy-able artifacts and currently the 
whole build might take 30 mins if a fresh clone.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

[RESULT] [VOTE] Release Apache Maven Shared Component: Maven Filtering 3.1.1 (Take 2)

2016-06-02 Thread Karl Heinz Marbaise 

Hi,

The vote has passed with the following result:

+1 : Hervé Boutemy, Anders Hammar, Tibor Digana, Karl Heinz Marbaise

PMC quorum: reached.

I will promote the artifacts to the central repo.

Kind regards
Karl Heinz Marbaise

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

[RESULT] [VOTE] Release Apache Maven Shared Component: Maven Shared Utils 3.0.1

2016-06-02 Thread Karl Heinz Marbaise 

Hi,

The vote has passed with the following result:

+1 : Hervé Boutemy, Petar Tahchiev, Michael Osipov, Karl Heinz Marbaise

PMC quorum: reached.

I will promote the artifacts to the central repo.

Kind regards
Karl Heinz Marbaise

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: [VOTE] Release Apache Maven Shared Component: Maven Filtering 3.1.1 (Take 2)

2016-06-02 Thread Tibor Digana 
+1

On Mon, May 30, 2016 at 10:37 PM, Karl Heinz Marbaise 
wrote:

> Hi,
>
> We have solved 6 issues:
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20MSHARED%20AND%20fixVersion%20%3D%20maven-filtering-3.1.1
>
> There are still a couple of issues left in JIRA:
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20MSHARED%20AND%20component%20%3D%20maven-filtering%20AND%20status%20%3D%20Open%20ORDER%20BY%20priority%20DESC
>
> Staging repo:
> https://repository.apache.org/content/repositories/maven-1272
>
> https://repository.apache.org/content/repositories/maven-1272/org/apache/maven/shared/maven-filtering/3.1.1/maven-filtering-3.1.1-source-release.zip
>
> Source release checksum(s):
> maven-filtering-3.1.1-source-release.zip sha1:
> f09258f2a7db1c2658c9898f0796abfd32fd9142
>
> Staging site:
> http://maven.apache.org/shared-archives/maven-filtering-LATEST/index.html
>
> Guide to testing staged releases:
> http://maven.apache.org/guides/development/guide-testing-releases.html
>
> Vote open for 72 hours.
>
> [ ] +1
> [ ] +0
> [ ] -1
>
> Kind regards
> Karl Heinz Marbaise
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
>

Re: [VOTE] Release Apache Maven Shared Component: Maven Filtering 3.1.1 (Take 2)

2016-06-02 Thread Anders Hammar 
+1

/Anders

On Mon, May 30, 2016 at 10:37 PM, Karl Heinz Marbaise 
wrote:

> Hi,
>
> We have solved 6 issues:
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20MSHARED%20AND%20fixVersion%20%3D%20maven-filtering-3.1.1
>
> There are still a couple of issues left in JIRA:
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20MSHARED%20AND%20component%20%3D%20maven-filtering%20AND%20status%20%3D%20Open%20ORDER%20BY%20priority%20DESC
>
> Staging repo:
> https://repository.apache.org/content/repositories/maven-1272
>
> https://repository.apache.org/content/repositories/maven-1272/org/apache/maven/shared/maven-filtering/3.1.1/maven-filtering-3.1.1-source-release.zip
>
> Source release checksum(s):
> maven-filtering-3.1.1-source-release.zip sha1:
> f09258f2a7db1c2658c9898f0796abfd32fd9142
>
> Staging site:
> http://maven.apache.org/shared-archives/maven-filtering-LATEST/index.html
>
> Guide to testing staged releases:
> http://maven.apache.org/guides/development/guide-testing-releases.html
>
> Vote open for 72 hours.
>
> [ ] +1
> [ ] +0
> [ ] -1
>
> Kind regards
> Karl Heinz Marbaise
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
>

[GitHub] maven issue #81: Hook up gossip as slf4j backend and implement customized co...

2016-06-02 Thread jdillon 
Github user jdillon commented on the issue:

https://github.com/apache/maven/pull/81
  
@jvanzyl I believe the output issues were sorted, but was still seeing some 
odd failures with what looked like resolution mismatches, downloading a version 
of one artifact but ending up with a slightly different version.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

[GitHub] maven-wagon issue #22: Implementation for Secure FTP (FTPS protocol)

2016-06-02 Thread michael-o 
Github user michael-o commented on the issue:

https://github.com/apache/maven-wagon/pull/22
  
@wiiitek The code snippet was just an example whow it could be solved for 
`ftps`. If updating Commons Net simply solves the issue, why not? I don't have 
any FTPS servers at hand to test your changes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

[GitHub] maven-wagon issue #22: Implementation for Secure FTP (FTPS protocol)

2016-06-02 Thread wiiitek 
Github user wiiitek commented on the issue:

https://github.com/apache/maven-wagon/pull/22
  
Hi, @michael-o, thank you for your reply,

as I understand wagon-http provider could recognize https protocol and use 
it when its needed?
But I am using **wagon-ftp** provider for uploading the site to remote 
location using:



some-id
ftp://example.com/some/path



I have seen [in the 
code](https://github.com/apache/maven-wagon/blob/wagon-2.10/wagon-providers/wagon-ftp/src/main/java/org/apache/maven/wagon/providers/ftp/FtpWagon.java#L115)
 that the default constructor for FTPClient is used. If we upgrade to 
commons-net in version 3.5, then explicit TLS is used by default.

But what about commons-net version 3.1? Is it secure to use wagon-ftp 
provider with `ftp:` protocol? May we use `ftps:` in url?

Could we have the change in 
[wagon-providers/wagon-ftp/src/site/apt/index.apt](https://github.com/apache/maven-wagon/blob/wagon-2.10/wagon-providers/wagon-ftp/src/site/apt/index.apt)
 to have it documented?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

[GitHub] maven issue #81: Hook up gossip as slf4j backend and implement customized co...

2016-06-02 Thread jvanzyl 
Github user jvanzyl commented on the issue:

https://github.com/apache/maven/pull/81
  
How are the ITs looking now? I know there were a few issues with the output 
not matching slf4j-simple that you were trying to sort out.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: any ideas why 'Scanning for projects' is very slow ?

2016-06-02 Thread Jorg Heymans 
Thanks Stephen, you know i actually considered putting this link in my
previous reply about why i am not using the freestyle job type :-) But
there is just too much that does not work anymore in freestyle, such as
- executor-local repositories
- maven releases
- automatic post-build deploy

... just to name a few. I am sure I could script all these things back into
a freestyle job but why ? I did attempt this several times over the years,
but ended up abandoning it each time because of the complexity i would need
to maintain myself + the several hundreds of build jobs to migrate ...

Jorg

On Thu, Jun 2, 2016 at 11:25 AM Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:

>
> http://javaadventure.blogspot.ie/2013/11/jenkins-maven-job-type-considered-evil.html
>
> On 2 June 2016 at 07:22, Jorg Heymans  wrote:
>
> > Hi,
> >
> > Thanks for the suggestions.
> >
> > --no-snapshot-updates does not make a difference. I tried out the
> profiler
> > and it gives me a breakdown of timings per phase, but it does not have
> any
> > timings on what happened before the first phase starts, which is exactly
> > where the slowdown occurs.
> >
> > Converting the job to a freestyle project in jenkins also solves the
> issue,
> > but it's not really an option for us since we rely on a lot of the
> > functionality that the maven job type offers. I guess i will have to head
> > back to the Jenkins side then to find out what is going on. I know
> Jenkins
> > is instrumenting the maven process with some extra functionality and even
> > rewrites the pom internally before execution.
> >
> > Apart from that, is there any other way to get a bit more debug info out
> of
> > maven during the 'scanning for projects' phase ?
> >
> > Jorg
> >
> > On Wed, Jun 1, 2016 at 8:24 PM Karl Heinz Marbaise 
> > wrote:
> >
> > > BTW: What also comes to my mind.
> > >
> > >
> > > Does the same happen if you ran that project as a freestyle project
> > > instead of Maven Job type (which i assume) ?
> > >
> > >
> > > Kind regards
> > > Karl Heinz Marbaise
> > > On 6/1/16 8:04 PM, Karl Heinz Marbaise wrote:
> > > > Hi,
> > > >
> > > > On 6/1/16 10:47 AM, Jorg Heymans wrote:
> > > >> Hi,
> > > >>
> > > >> I am trying to pinpoint a performance issue we are incurring with
> > maven
> > > >> under Jenkins. Basically what we are seeing is that maven is taking
> a
> > > >> very
> > > >> long time getting past the initial 'Scanning for projects' message.
> As
> > > >> you
> > > >> can see in below snippet, more than half a minute elapses before the
> > > >> first
> > > >> module is starting to build.
> > > >>
> > > >> *00:00:11.634* Maven home:
> > > >>
> > >
> >
> /home/ci/jenkins/data/tools/hudson.tasks.Maven_MavenInstallation/maven3*00:00:11.634*
> > > >>
> > > >> Java version: 1.7.0_80, vendor: Oracle Corporation*00:00:11.634*
> Java
> > > >> home:
> > > >>
> /home/ci/jenkins/data/tools/hudson.model.JDK/jdk1.7/jre*00:00:11.634*
> > > >> Default locale: en, platform encoding: ISO646-US*00:00:11.634* OS
> > > >> name: "sunos", version: "5.10", arch: "sparc", family: "unix"
> > > >
> > > >
> > > >>
> > > >> *00:00:11.757* [INFO] Scanning for projects...
> > > >>
> > > >> *00:00:42.350* [INFO]
> > > >>*00:00:42.350* [INFO]
> > > >>
> > >
> >
> *00:00:42.350*
> > > >>
> > > >> [INFO] Building Document Implementation 0.2-SNAPSHOT
> > > >
> > > > Is this project an Maven Tycho project ?
> > > >
> > > > Kind regards
> > > > Karl Heinz Marbaise
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > For additional commands, e-mail: dev-h...@maven.apache.org
> > >
> > >
> >
>

Re: any ideas why 'Scanning for projects' is very slow ?

2016-06-02 Thread Stephen Connolly 
http://javaadventure.blogspot.ie/2013/11/jenkins-maven-job-type-considered-evil.html

On 2 June 2016 at 07:22, Jorg Heymans  wrote:

> Hi,
>
> Thanks for the suggestions.
>
> --no-snapshot-updates does not make a difference. I tried out the profiler
> and it gives me a breakdown of timings per phase, but it does not have any
> timings on what happened before the first phase starts, which is exactly
> where the slowdown occurs.
>
> Converting the job to a freestyle project in jenkins also solves the issue,
> but it's not really an option for us since we rely on a lot of the
> functionality that the maven job type offers. I guess i will have to head
> back to the Jenkins side then to find out what is going on. I know Jenkins
> is instrumenting the maven process with some extra functionality and even
> rewrites the pom internally before execution.
>
> Apart from that, is there any other way to get a bit more debug info out of
> maven during the 'scanning for projects' phase ?
>
> Jorg
>
> On Wed, Jun 1, 2016 at 8:24 PM Karl Heinz Marbaise 
> wrote:
>
> > BTW: What also comes to my mind.
> >
> >
> > Does the same happen if you ran that project as a freestyle project
> > instead of Maven Job type (which i assume) ?
> >
> >
> > Kind regards
> > Karl Heinz Marbaise
> > On 6/1/16 8:04 PM, Karl Heinz Marbaise wrote:
> > > Hi,
> > >
> > > On 6/1/16 10:47 AM, Jorg Heymans wrote:
> > >> Hi,
> > >>
> > >> I am trying to pinpoint a performance issue we are incurring with
> maven
> > >> under Jenkins. Basically what we are seeing is that maven is taking a
> > >> very
> > >> long time getting past the initial 'Scanning for projects' message. As
> > >> you
> > >> can see in below snippet, more than half a minute elapses before the
> > >> first
> > >> module is starting to build.
> > >>
> > >> *00:00:11.634* Maven home:
> > >>
> >
> /home/ci/jenkins/data/tools/hudson.tasks.Maven_MavenInstallation/maven3*00:00:11.634*
> > >>
> > >> Java version: 1.7.0_80, vendor: Oracle Corporation*00:00:11.634* Java
> > >> home:
> > >> /home/ci/jenkins/data/tools/hudson.model.JDK/jdk1.7/jre*00:00:11.634*
> > >> Default locale: en, platform encoding: ISO646-US*00:00:11.634* OS
> > >> name: "sunos", version: "5.10", arch: "sparc", family: "unix"
> > >
> > >
> > >>
> > >> *00:00:11.757* [INFO] Scanning for projects...
> > >>
> > >> *00:00:42.350* [INFO]
> > >>*00:00:42.350* [INFO]
> > >>
> >
> *00:00:42.350*
> > >>
> > >> [INFO] Building Document Implementation 0.2-SNAPSHOT
> > >
> > > Is this project an Maven Tycho project ?
> > >
> > > Kind regards
> > > Karl Heinz Marbaise
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > For additional commands, e-mail: dev-h...@maven.apache.org
> >
> >
>

Re: Maven Memory Consumption

2016-06-02 Thread Tamás Cservenák 
Olivier, if you refer to the slf4j-gossip branch, that IMHO Jason's PR
supersedes it.
Will drop that branch.

On Thu, Jun 2, 2016 at 8:38 AM Olivier Lamy  wrote:

> well I think this color stuff has already been done differently but never
> accepted..
>
> On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:
>
> > another feature that would be great for this release:
> > https://github.com/apache/maven/pull/81
> >
> > I still didn't have time to work on it, but I like the screenshot
> > The only thing that I'd like to check is: is tty detection working? ie
> does
> > color automatically disappear if there is no tty?
> >
> > Regards,
> >
> > Hervé
> >
> > Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :
> > > +1
> > > this is something that was often seen: this is great that it is fixed!
> > >
> > > For example, the last time I published Maven core site, the build
> simply
> > > failed because of PermgenSpace: now it is working like a charm...
> > >
> > > This release will be a must!
> > >
> > > Regards,
> > >
> > > Hervé
> > >
> > > Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
> > > > Hi Manfred,
> > > >
> > > > On 6/1/16 12:24 AM, Manfred Moser wrote:
> > > > > I can feel your excitement coming through in the emails.. ;-)
> > > >
> > > > Of course I'm excited ;-)
> > > >
> > > > ...cause it's very important...I have had heard many customers saying
> > > > they will not upgrade to newer versions of Maven exactly based on
> such
> > > > issue(s)...
> > > >
> > > > which is in general bad ...
> > > >
> > > >
> > > > This will break their argument ;-)...
> > > >
> > > > > Karl Heinz Marbaise wrote on 2016-05-31 15:14:
> > > > >> Hi,
> > > > >>
> > > > >> tested without the patch (-Xmx6g) ...run time for the test project
> > more
> > > > >> than two 2 Minutes
> > > > >>
> > > > >> running with the patch (-Xmx1g):
> > > > >>
> > > > >> Run time ca. 27 seconds...
> > > > >>
> > > > >> also worked with -Xmx768m ...ca. 30 seconds...
> > > > >>
> > > > >> so looks very good...
> > > > >>
> > > > >> Let us wait what the IT's say...
> > > > >>
> > > > >> Kind regards
> > > > >> Karl Heinz Marbaise
> > > > >>
> > > > >> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> > > > >>> Hi,
> > > > >>>
> > > > >>> after more investigation and an extremly good tip of
> Andriy...(see
> > > > >>> MNG-6030) and in the end the solution:
> > > > >>>
> > > > >>> Using test project with 5000 modules just doing:
> > > > >>>
> > > > >>> mvn clean
> > > > >>>
> > > > >>> using the patch now in master
> > > > >>> (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> > > > >>> extremely reduces the memory footprint...
> > > > >>>
> > > > >>>
> > https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > > > >>> /M
> > > > >>> aven340-with-patch-5000.png
> > > > >>>
> > > > >>>
> > > > >>> This shows the result using the patch
> > > > >>>
> > > > >>> The following shows Maven 3.3.9:
> > > > >>>
> > > > >>>
> > https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > > > >>> /M
> > > > >>> aven339-5000.png
> > > > >>>
> > > > >>>
> > > > >>> Many thanks to Andriy for the support and help...
> > > > >>>
> > > > >>> we will see if not IT's will fail on the change.
> > > > >>>
> > > > >>>
> > > > >>> Kind regards
> > > > >>> Karl Heinz Marbaise
> > > > >>>
> > > > >>> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
> > > >  Hi,
> > > > 
> > > >  i started a little bit more detailed analysis..
> > > > 
> > > >  very simple via JConsole and running the different versions...
> > > > 
> > > >  I have summarized this here:
> > > > 
> > > >  https://github.com/khmarbaise/maven-test-project-generator
> > > > 
> > > >  Kind regards
> > > >  Karl Heinz Marbaise
> > > > 
> > > >  On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
> > > > > Hi to all,
> > > > >
> > > > > i have a question concerning the memory consumption...
> > > > >
> > > > > If i run maven with the same JDK and the same reactor and build
> > with
> > > > > the
> > > > > same parameter and plugins...
> > > > >
> > > > > will the printout at the end of the build (Final Memory)
> > something
> > > > > realiable about the consumption of the JVM during the build
> > ?...Or
> > > > > is
> > > > > it
> > > > > at least a hint...or would i need to do something different
> (BTW:
> > > > > Someone has a hint about that?) ...
> > > > >
> > > > >
> > > > > [INFO] BUILD SUCCESS
> > > > > [INFO]
> > > > >
> > 
> > > > > --
> > > > > --
> > > > > [INFO] Total time: 6.431 s
> > > > > [INFO] Finished at: 2016-04-17T17:46:58+02:00
> > > > > [INFO] Final Memory: 47M/638M
> > > > >
> > > > > So if i ran the same build with different Maven versions so
> could
> > > > > this
> > > > > give

Re: Maven Memory Consumption

2016-06-02 Thread Olivier Lamy 
well I think this color stuff has already been done differently but never
accepted..

On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:

> another feature that would be great for this release:
> https://github.com/apache/maven/pull/81
>
> I still didn't have time to work on it, but I like the screenshot
> The only thing that I'd like to check is: is tty detection working? ie does
> color automatically disappear if there is no tty?
>
> Regards,
>
> Hervé
>
> Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :
> > +1
> > this is something that was often seen: this is great that it is fixed!
> >
> > For example, the last time I published Maven core site, the build simply
> > failed because of PermgenSpace: now it is working like a charm...
> >
> > This release will be a must!
> >
> > Regards,
> >
> > Hervé
> >
> > Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
> > > Hi Manfred,
> > >
> > > On 6/1/16 12:24 AM, Manfred Moser wrote:
> > > > I can feel your excitement coming through in the emails.. ;-)
> > >
> > > Of course I'm excited ;-)
> > >
> > > ...cause it's very important...I have had heard many customers saying
> > > they will not upgrade to newer versions of Maven exactly based on such
> > > issue(s)...
> > >
> > > which is in general bad ...
> > >
> > >
> > > This will break their argument ;-)...
> > >
> > > > Karl Heinz Marbaise wrote on 2016-05-31 15:14:
> > > >> Hi,
> > > >>
> > > >> tested without the patch (-Xmx6g) ...run time for the test project
> more
> > > >> than two 2 Minutes
> > > >>
> > > >> running with the patch (-Xmx1g):
> > > >>
> > > >> Run time ca. 27 seconds...
> > > >>
> > > >> also worked with -Xmx768m ...ca. 30 seconds...
> > > >>
> > > >> so looks very good...
> > > >>
> > > >> Let us wait what the IT's say...
> > > >>
> > > >> Kind regards
> > > >> Karl Heinz Marbaise
> > > >>
> > > >> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> > > >>> Hi,
> > > >>>
> > > >>> after more investigation and an extremly good tip of Andriy...(see
> > > >>> MNG-6030) and in the end the solution:
> > > >>>
> > > >>> Using test project with 5000 modules just doing:
> > > >>>
> > > >>> mvn clean
> > > >>>
> > > >>> using the patch now in master
> > > >>> (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> > > >>> extremely reduces the memory footprint...
> > > >>>
> > > >>>
> https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > > >>> /M
> > > >>> aven340-with-patch-5000.png
> > > >>>
> > > >>>
> > > >>> This shows the result using the patch
> > > >>>
> > > >>> The following shows Maven 3.3.9:
> > > >>>
> > > >>>
> https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > > >>> /M
> > > >>> aven339-5000.png
> > > >>>
> > > >>>
> > > >>> Many thanks to Andriy for the support and help...
> > > >>>
> > > >>> we will see if not IT's will fail on the change.
> > > >>>
> > > >>>
> > > >>> Kind regards
> > > >>> Karl Heinz Marbaise
> > > >>>
> > > >>> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
> > >  Hi,
> > > 
> > >  i started a little bit more detailed analysis..
> > > 
> > >  very simple via JConsole and running the different versions...
> > > 
> > >  I have summarized this here:
> > > 
> > >  https://github.com/khmarbaise/maven-test-project-generator
> > > 
> > >  Kind regards
> > >  Karl Heinz Marbaise
> > > 
> > >  On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
> > > > Hi to all,
> > > >
> > > > i have a question concerning the memory consumption...
> > > >
> > > > If i run maven with the same JDK and the same reactor and build
> with
> > > > the
> > > > same parameter and plugins...
> > > >
> > > > will the printout at the end of the build (Final Memory)
> something
> > > > realiable about the consumption of the JVM during the build
> ?...Or
> > > > is
> > > > it
> > > > at least a hint...or would i need to do something different (BTW:
> > > > Someone has a hint about that?) ...
> > > >
> > > >
> > > > [INFO] BUILD SUCCESS
> > > > [INFO]
> > > >
> 
> > > > --
> > > > --
> > > > [INFO] Total time: 6.431 s
> > > > [INFO] Finished at: 2016-04-17T17:46:58+02:00
> > > > [INFO] Final Memory: 47M/638M
> > > >
> > > > So if i ran the same build with different Maven versions so could
> > > > this
> > > > give us a hint where more memory is consumed ...(to identify
> where
> > > > and
> > > > why is a different story)...
> > > >
> > > > Kind regards
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > For additional commands, e-mail: dev-h...@maven.apache.org
> >
> > -
> > To unsubscribe, e-mail:

Re: Maven Memory Consumption

2016-06-02 Thread Hervé BOUTEMY 
another feature that would be great for this release:
https://github.com/apache/maven/pull/81

I still didn't have time to work on it, but I like the screenshot
The only thing that I'd like to check is: is tty detection working? ie does 
color automatically disappear if there is no tty?

Regards,

Hervé

Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :
> +1
> this is something that was often seen: this is great that it is fixed!
> 
> For example, the last time I published Maven core site, the build simply
> failed because of PermgenSpace: now it is working like a charm...
> 
> This release will be a must!
> 
> Regards,
> 
> Hervé
> 
> Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
> > Hi Manfred,
> > 
> > On 6/1/16 12:24 AM, Manfred Moser wrote:
> > > I can feel your excitement coming through in the emails.. ;-)
> > 
> > Of course I'm excited ;-)
> > 
> > ...cause it's very important...I have had heard many customers saying
> > they will not upgrade to newer versions of Maven exactly based on such
> > issue(s)...
> > 
> > which is in general bad ...
> > 
> > 
> > This will break their argument ;-)...
> > 
> > > Karl Heinz Marbaise wrote on 2016-05-31 15:14:
> > >> Hi,
> > >> 
> > >> tested without the patch (-Xmx6g) ...run time for the test project more
> > >> than two 2 Minutes
> > >> 
> > >> running with the patch (-Xmx1g):
> > >> 
> > >> Run time ca. 27 seconds...
> > >> 
> > >> also worked with -Xmx768m ...ca. 30 seconds...
> > >> 
> > >> so looks very good...
> > >> 
> > >> Let us wait what the IT's say...
> > >> 
> > >> Kind regards
> > >> Karl Heinz Marbaise
> > >> 
> > >> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> > >>> Hi,
> > >>> 
> > >>> after more investigation and an extremly good tip of Andriy...(see
> > >>> MNG-6030) and in the end the solution:
> > >>> 
> > >>> Using test project with 5000 modules just doing:
> > >>> 
> > >>> mvn clean
> > >>> 
> > >>> using the patch now in master
> > >>> (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> > >>> extremely reduces the memory footprint...
> > >>> 
> > >>> https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > >>> /M
> > >>> aven340-with-patch-5000.png
> > >>> 
> > >>> 
> > >>> This shows the result using the patch
> > >>> 
> > >>> The following shows Maven 3.3.9:
> > >>> 
> > >>> https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > >>> /M
> > >>> aven339-5000.png
> > >>> 
> > >>> 
> > >>> Many thanks to Andriy for the support and help...
> > >>> 
> > >>> we will see if not IT's will fail on the change.
> > >>> 
> > >>> 
> > >>> Kind regards
> > >>> Karl Heinz Marbaise
> > >>> 
> > >>> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
> >  Hi,
> >  
> >  i started a little bit more detailed analysis..
> >  
> >  very simple via JConsole and running the different versions...
> >  
> >  I have summarized this here:
> >  
> >  https://github.com/khmarbaise/maven-test-project-generator
> >  
> >  Kind regards
> >  Karl Heinz Marbaise
> >  
> >  On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
> > > Hi to all,
> > > 
> > > i have a question concerning the memory consumption...
> > > 
> > > If i run maven with the same JDK and the same reactor and build with
> > > the
> > > same parameter and plugins...
> > > 
> > > will the printout at the end of the build (Final Memory) something
> > > realiable about the consumption of the JVM during the build ?...Or
> > > is
> > > it
> > > at least a hint...or would i need to do something different (BTW:
> > > Someone has a hint about that?) ...
> > > 
> > > 
> > > [INFO] BUILD SUCCESS
> > > [INFO]
> > > 
> > > --
> > > --
> > > [INFO] Total time: 6.431 s
> > > [INFO] Finished at: 2016-04-17T17:46:58+02:00
> > > [INFO] Final Memory: 47M/638M
> > > 
> > > So if i ran the same build with different Maven versions so could
> > > this
> > > give us a hint where more memory is consumed ...(to identify where
> > > and
> > > why is a different story)...
> > > 
> > > Kind regards
> > 
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > For additional commands, e-mail: dev-h...@maven.apache.org
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Maven Memory Consumption

2016-06-02 Thread Hervé BOUTEMY 
+1
this is something that was often seen: this is great that it is fixed!

For example, the last time I published Maven core site, the build simply 
failed because of PermgenSpace: now it is working like a charm...

This release will be a must!

Regards,

Hervé

Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
> Hi Manfred,
> 
> On 6/1/16 12:24 AM, Manfred Moser wrote:
> > I can feel your excitement coming through in the emails.. ;-)
> 
> Of course I'm excited ;-)
> 
> ...cause it's very important...I have had heard many customers saying
> they will not upgrade to newer versions of Maven exactly based on such
> issue(s)...
> 
> which is in general bad ...
> 
> 
> This will break their argument ;-)...
> 
> > Karl Heinz Marbaise wrote on 2016-05-31 15:14:
> >> Hi,
> >> 
> >> tested without the patch (-Xmx6g) ...run time for the test project more
> >> than two 2 Minutes
> >> 
> >> running with the patch (-Xmx1g):
> >> 
> >> Run time ca. 27 seconds...
> >> 
> >> also worked with -Xmx768m ...ca. 30 seconds...
> >> 
> >> so looks very good...
> >> 
> >> Let us wait what the IT's say...
> >> 
> >> Kind regards
> >> Karl Heinz Marbaise
> >> 
> >> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> >>> Hi,
> >>> 
> >>> after more investigation and an extremly good tip of Andriy...(see
> >>> MNG-6030) and in the end the solution:
> >>> 
> >>> Using test project with 5000 modules just doing:
> >>> 
> >>> mvn clean
> >>> 
> >>> using the patch now in master (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> >>> extremely reduces the memory footprint...
> >>> 
> >>> https://github.com/khmarbaise/maven-test-project-generator/blob/master/M
> >>> aven340-with-patch-5000.png
> >>> 
> >>> 
> >>> This shows the result using the patch
> >>> 
> >>> The following shows Maven 3.3.9:
> >>> 
> >>> https://github.com/khmarbaise/maven-test-project-generator/blob/master/M
> >>> aven339-5000.png
> >>> 
> >>> 
> >>> Many thanks to Andriy for the support and help...
> >>> 
> >>> we will see if not IT's will fail on the change.
> >>> 
> >>> 
> >>> Kind regards
> >>> Karl Heinz Marbaise
> >>> 
> >>> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
>  Hi,
>  
>  i started a little bit more detailed analysis..
>  
>  very simple via JConsole and running the different versions...
>  
>  I have summarized this here:
>  
>  https://github.com/khmarbaise/maven-test-project-generator
>  
>  Kind regards
>  Karl Heinz Marbaise
>  
>  On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
> > Hi to all,
> > 
> > i have a question concerning the memory consumption...
> > 
> > If i run maven with the same JDK and the same reactor and build with
> > the
> > same parameter and plugins...
> > 
> > will the printout at the end of the build (Final Memory) something
> > realiable about the consumption of the JVM during the build ?...Or is
> > it
> > at least a hint...or would i need to do something different (BTW:
> > Someone has a hint about that?) ...
> > 
> > 
> > [INFO] BUILD SUCCESS
> > [INFO]
> > --
> > --
> > [INFO] Total time: 6.431 s
> > [INFO] Finished at: 2016-04-17T17:46:58+02:00
> > [INFO] Final Memory: 47M/638M
> > 
> > So if i ran the same build with different Maven versions so could this
> > give us a hint where more memory is consumed ...(to identify where and
> > why is a different story)...
> > 
> > Kind regards
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: any ideas why 'Scanning for projects' is very slow ?

2016-06-02 Thread Jorg Heymans 
Hi,

Thanks for the suggestions.

--no-snapshot-updates does not make a difference. I tried out the profiler
and it gives me a breakdown of timings per phase, but it does not have any
timings on what happened before the first phase starts, which is exactly
where the slowdown occurs.

Converting the job to a freestyle project in jenkins also solves the issue,
but it's not really an option for us since we rely on a lot of the
functionality that the maven job type offers. I guess i will have to head
back to the Jenkins side then to find out what is going on. I know Jenkins
is instrumenting the maven process with some extra functionality and even
rewrites the pom internally before execution.

Apart from that, is there any other way to get a bit more debug info out of
maven during the 'scanning for projects' phase ?

Jorg

On Wed, Jun 1, 2016 at 8:24 PM Karl Heinz Marbaise 
wrote:

> BTW: What also comes to my mind.
>
>
> Does the same happen if you ran that project as a freestyle project
> instead of Maven Job type (which i assume) ?
>
>
> Kind regards
> Karl Heinz Marbaise
> On 6/1/16 8:04 PM, Karl Heinz Marbaise wrote:
> > Hi,
> >
> > On 6/1/16 10:47 AM, Jorg Heymans wrote:
> >> Hi,
> >>
> >> I am trying to pinpoint a performance issue we are incurring with maven
> >> under Jenkins. Basically what we are seeing is that maven is taking a
> >> very
> >> long time getting past the initial 'Scanning for projects' message. As
> >> you
> >> can see in below snippet, more than half a minute elapses before the
> >> first
> >> module is starting to build.
> >>
> >> *00:00:11.634* Maven home:
> >>
> /home/ci/jenkins/data/tools/hudson.tasks.Maven_MavenInstallation/maven3*00:00:11.634*
> >>
> >> Java version: 1.7.0_80, vendor: Oracle Corporation*00:00:11.634* Java
> >> home:
> >> /home/ci/jenkins/data/tools/hudson.model.JDK/jdk1.7/jre*00:00:11.634*
> >> Default locale: en, platform encoding: ISO646-US*00:00:11.634* OS
> >> name: "sunos", version: "5.10", arch: "sparc", family: "unix"
> >
> >
> >>
> >> *00:00:11.757* [INFO] Scanning for projects...
> >>
> >> *00:00:42.350* [INFO]
> >>*00:00:42.350* [INFO]
> >>
> *00:00:42.350*
> >>
> >> [INFO] Building Document Implementation 0.2-SNAPSHOT
> >
> > Is this project an Maven Tycho project ?
> >
> > Kind regards
> > Karl Heinz Marbaise
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
>