Re: Maven memory consumption issue

[Hervé BOUTEMY]

as discusses in the Jira issue, the cache on Version Range is simple and very 
efficient

can someone review and second for merge to master?

Regards,

Hervé

Le mercredi 23 janvier 2019, 11:07:43 CET Hervé BOUTEMY a écrit :
> Hi,
> 
> Interesting.
> I created MNG-6571 to track this issue: please move the memory dump on this
> issue.
> Then I created MNG-6571 branch [2] with a cache on VersionRange: this class
> is currently immutable, then caching it is trivial. I don't know if it's
> the best location, or if there are not other locations where a cache would
> make sense, but it was an easy and safe try.
> 
> Can you build this branch for yourself and share a new memory dump, please?
> 
> Regards,
> 
> Hervé
> 
> [1] https://issues.apache.org/jira/browse/MNG-6571
> 
> [2] https://github.com/apache/maven/tree/MNG-6571
> 
> Le mercredi 23 janvier 2019, 10:10:40 CET Dameron, Yann a écrit :
> > Hi,
> > 
> > 
> > 
> > I'm working on a huge project with plenty modules and we are facing a
> > memory consumption issue. We observed it when Maven computes the reactor
> > and then during the build itself.
> > 
> > 
> > 
> > Quick summary of our situation (as we can't share the POMs)
> > 
> >   *   We have a dedicated POM to manage all versions (3rd parties and our
> > 
> > own modules). This POM is split in 5 sub poms to organize them a bit
> > (apache, netbeans, xxx, ...)
> > 
> > On this POM structure, we declare 3049 
> > 
> >   *   We have a parent pom. This one is used by all our 1628 modules. It
> > 
> > defines the plugins versions and import the dependency management
> > (explained above). *   We have 1628 modules which are using our parent pom
> > (and then inherit the dependency management imported by the parent) *   We
> > have an aggregator POMs structure which is responsible to include all the
> > modules. Root aggregator is the one on which the full rebuild is
> > triggered.
> > 
> > 
> > 
> > When Maven computes the reactor, it consumes a huge amount of memory (8 -
> > 12 Gb)
> > 
> > 
> > 
> > I made a memory dump just before the end of reactor computation. I
> > attached
> > a screenshot in MNG-5661
> > as
> > I feel it's related, please remove it if it's not appropriated. On this
> > screenshot we clearly see from where come from our memory issues:
> > https://issues.apache.org/jira/secure/attachment/12955925/Maven-Reactor-Du
> > m
> > p.png
> > 
> > 
> > 
> > The number of Dependency instance seems to be related to our structure.
> > All
> > our projects import our big dependency management (inherited from the
> > parent) so we have 1628 x 3049 => ~ 5 000 000 instances.
> > 
> > This is also approximately the number of MavenProject, DefaultArtifact,
> > VersionRange, etc... so I feel that the memory consumption comes from the
> > fact that Maven doesn't detect such cases and duplicate everything.
> > 
> > 
> > 
> > Is Maven optimized to detect such cases? It would be great to ensure that
> > Dependency (and others) instances are not duplicated.
> > 
> > 
> > 
> > Thanks,
> > Yann
> > 
> > Information in this e-mail and any attachments is confidential, and may
> > not
> > be copied or used by anyone other than the addressee, nor disclosed to any
> > third party without our permission. There is no intention to create any
> > legally binding contract or other binding commitment through the use of
> > this electronic communication unless it is issued in accordance with the
> > Experian Limited standard terms and conditions of purchase or other
> > express
> > written agreement between Experian Limited and the recipient. Although
> > Experian has taken reasonable steps to ensure that this communication and
> > any attachments are free from computer viruses, you are advised to take
> > your own steps to ensure that they are actually virus free.
> > 
> > Experian Ltd is authorised and regulated by the Financial Conduct
> > Authority. Companies Act information: Registered name: Experian Limited.
> > Registered office: The Sir John Peace Building, Experian Way, NG2
> > Business Park, Nottingham, NG80 1ZZ, United Kingdom. Place of
> > registration: England and Wales. Registered number: 653331.
> > 
> > Vos informations nominatives sont exploitées par la société Scorex S.A.M.
> > (Groupe Experian)  dans le cadre du traitement ayant pour finalité
> > "Gestion
> > de la messagerie électronique professionnelle". Conformément à la loi n°
> > 1.165, modifiée, vous disposez d'un droit d'accès, de rectification et de
> > suppression en écrivant au Secrétariat Général de la société Scorex, sise,
> > 2, rue de la Lüjerneta à Monaco (98000).
> > 
> > Your personal information is used by Scorex S.A.M. (Experian Group) as
> > part
> > of the processing for the purpose of "Professional e-mail management". In
> > accordance with the law n ° 1.165, modified, you have a right of access,
> > correction and suppression by writing to the General Secretariat of the
> > 

Re: Maven memory consumption issue

[Yann Dameron]

I'm still computing the retained references but the first results look great.

Dump file once reactor is computed moved from 14Gb to 7.4Gb (I didn't let the 
build run to the end).
Instances of VersionRange moved from 5 million to 6525. The cache has a 
positive impact on the String instances, BigInteger instances, etc... which 
lead to a massive gain in memory. Well done !

We still have a 4.7 million instances of Dependency and DefaultArtifact but 
looking at the patch it was expected. 

As soon as I got the retained references computed, I'll attach the new 
screenshot of the dump to MNG-6571. 

On 2019/01/23 10:07:43, Hervé BOUTEMY  wrote: 
> Hi,
> 
> Interesting.
> I created MNG-6571 to track this issue: please move the memory dump on this 
> issue.
> Then I created MNG-6571 branch [2] with a cache on VersionRange: this class 
> is 
> currently immutable, then caching it is trivial. I don't know if it's the 
> best 
> location, or if there are not other locations where a cache would make sense, 
> but it was an easy and safe try.
> 
> Can you build this branch for yourself and share a new memory dump, please?
> 
> Regards,
> 
> Hervé
> 
> [1] https://issues.apache.org/jira/browse/MNG-6571
> 
> [2] https://github.com/apache/maven/tree/MNG-6571
> 
> Le mercredi 23 janvier 2019, 10:10:40 CET Dameron, Yann a écrit :
> > Hi,
> > 
> > 
> > 
> > I'm working on a huge project with plenty modules and we are facing a memory
> > consumption issue. We observed it when Maven computes the reactor and then
> > during the build itself.
> > 
> > 
> > 
> > Quick summary of our situation (as we can't share the POMs)
> > 
> >   *   We have a dedicated POM to manage all versions (3rd parties and our
> > own modules). This POM is split in 5 sub poms to organize them a bit
> > (apache, netbeans, xxx, ...)
> > 
> > On this POM structure, we declare 3049 
> > 
> >   *   We have a parent pom. This one is used by all our 1628 modules. It
> > defines the plugins versions and import the dependency management
> > (explained above). *   We have 1628 modules which are using our parent pom
> > (and then inherit the dependency management imported by the parent) *   We
> > have an aggregator POMs structure which is responsible to include all the
> > modules. Root aggregator is the one on which the full rebuild is triggered.
> > 
> > 
> > 
> > When Maven computes the reactor, it consumes a huge amount of memory (8 - 12
> > Gb)
> > 
> > 
> > 
> > I made a memory dump just before the end of reactor computation. I attached
> > a screenshot in MNG-5661 as
> > I feel it's related, please remove it if it's not appropriated. On this
> > screenshot we clearly see from where come from our memory issues:
> > https://issues.apache.org/jira/secure/attachment/12955925/Maven-Reactor-Dum
> > p.png
> > 
> > 
> > 
> > The number of Dependency instance seems to be related to our structure. All
> > our projects import our big dependency management (inherited from the
> > parent) so we have 1628 x 3049 => ~ 5 000 000 instances.
> > 
> > This is also approximately the number of MavenProject, DefaultArtifact,
> > VersionRange, etc... so I feel that the memory consumption comes from the
> > fact that Maven doesn't detect such cases and duplicate everything.
> > 
> > 
> > 
> > Is Maven optimized to detect such cases? It would be great to ensure that
> > Dependency (and others) instances are not duplicated.
> > 
> > 
> > 
> > Thanks,
> > Yann
> > 
> > Information in this e-mail and any attachments is confidential, and may not
> > be copied or used by anyone other than the addressee, nor disclosed to any
> > third party without our permission. There is no intention to create any
> > legally binding contract or other binding commitment through the use of
> > this electronic communication unless it is issued in accordance with the
> > Experian Limited standard terms and conditions of purchase or other express
> > written agreement between Experian Limited and the recipient. Although
> > Experian has taken reasonable steps to ensure that this communication and
> > any attachments are free from computer viruses, you are advised to take
> > your own steps to ensure that they are actually virus free.
> > 
> > Experian Ltd is authorised and regulated by the Financial Conduct Authority.
> > Companies Act information: Registered name: Experian Limited. Registered
> > office: The Sir John Peace Building, Experian Way, NG2 Business Park,
> > Nottingham, NG80 1ZZ, United Kingdom. Place of registration: England and
> > Wales. Registered number: 653331.
> > 
> > Vos informations nominatives sont exploitées par la société Scorex S.A.M.
> > (Groupe Experian)  dans le cadre du traitement ayant pour finalité "Gestion
> > de la messagerie électronique professionnelle". Conformément à la loi n°
> > 1.165, modifiée, vous disposez d'un droit d'accès, de rectification et de
> > suppression en écrivant au Secrétariat Général de la 

Re: Maven memory consumption issue

[Hervé BOUTEMY]

Hi,

Interesting.
I created MNG-6571 to track this issue: please move the memory dump on this 
issue.
Then I created MNG-6571 branch [2] with a cache on VersionRange: this class is 
currently immutable, then caching it is trivial. I don't know if it's the best 
location, or if there are not other locations where a cache would make sense, 
but it was an easy and safe try.

Can you build this branch for yourself and share a new memory dump, please?

Regards,

Hervé

[1] https://issues.apache.org/jira/browse/MNG-6571

[2] https://github.com/apache/maven/tree/MNG-6571

Le mercredi 23 janvier 2019, 10:10:40 CET Dameron, Yann a écrit :
> Hi,
> 
> 
> 
> I'm working on a huge project with plenty modules and we are facing a memory
> consumption issue. We observed it when Maven computes the reactor and then
> during the build itself.
> 
> 
> 
> Quick summary of our situation (as we can't share the POMs)
> 
>   *   We have a dedicated POM to manage all versions (3rd parties and our
> own modules). This POM is split in 5 sub poms to organize them a bit
> (apache, netbeans, xxx, ...)
> 
> On this POM structure, we declare 3049 
> 
>   *   We have a parent pom. This one is used by all our 1628 modules. It
> defines the plugins versions and import the dependency management
> (explained above). *   We have 1628 modules which are using our parent pom
> (and then inherit the dependency management imported by the parent) *   We
> have an aggregator POMs structure which is responsible to include all the
> modules. Root aggregator is the one on which the full rebuild is triggered.
> 
> 
> 
> When Maven computes the reactor, it consumes a huge amount of memory (8 - 12
> Gb)
> 
> 
> 
> I made a memory dump just before the end of reactor computation. I attached
> a screenshot in MNG-5661 as
> I feel it's related, please remove it if it's not appropriated. On this
> screenshot we clearly see from where come from our memory issues:
> https://issues.apache.org/jira/secure/attachment/12955925/Maven-Reactor-Dum
> p.png
> 
> 
> 
> The number of Dependency instance seems to be related to our structure. All
> our projects import our big dependency management (inherited from the
> parent) so we have 1628 x 3049 => ~ 5 000 000 instances.
> 
> This is also approximately the number of MavenProject, DefaultArtifact,
> VersionRange, etc... so I feel that the memory consumption comes from the
> fact that Maven doesn't detect such cases and duplicate everything.
> 
> 
> 
> Is Maven optimized to detect such cases? It would be great to ensure that
> Dependency (and others) instances are not duplicated.
> 
> 
> 
> Thanks,
> Yann
> 
> Information in this e-mail and any attachments is confidential, and may not
> be copied or used by anyone other than the addressee, nor disclosed to any
> third party without our permission. There is no intention to create any
> legally binding contract or other binding commitment through the use of
> this electronic communication unless it is issued in accordance with the
> Experian Limited standard terms and conditions of purchase or other express
> written agreement between Experian Limited and the recipient. Although
> Experian has taken reasonable steps to ensure that this communication and
> any attachments are free from computer viruses, you are advised to take
> your own steps to ensure that they are actually virus free.
> 
> Experian Ltd is authorised and regulated by the Financial Conduct Authority.
> Companies Act information: Registered name: Experian Limited. Registered
> office: The Sir John Peace Building, Experian Way, NG2 Business Park,
> Nottingham, NG80 1ZZ, United Kingdom. Place of registration: England and
> Wales. Registered number: 653331.
> 
> Vos informations nominatives sont exploitées par la société Scorex S.A.M.
> (Groupe Experian)  dans le cadre du traitement ayant pour finalité "Gestion
> de la messagerie électronique professionnelle". Conformément à la loi n°
> 1.165, modifiée, vous disposez d'un droit d'accès, de rectification et de
> suppression en écrivant au Secrétariat Général de la société Scorex, sise,
> 2, rue de la Lüjerneta à Monaco (98000).
> 
> Your personal information is used by Scorex S.A.M. (Experian Group) as part
> of the processing for the purpose of "Professional e-mail management". In
> accordance with the law n ° 1.165, modified, you have a right of access,
> correction and suppression by writing to the General Secretariat of the
> company, located, 2, rue de la Lüjerneta in Monaco (98000).





-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Maven memory consumption issue

[Dameron, Yann]

Hi,



I'm working on a huge project with plenty modules and we are facing a memory 
consumption issue. We observed it when Maven computes the reactor and then 
during the build itself.



Quick summary of our situation (as we can't share the POMs)

  *   We have a dedicated POM to manage all versions (3rd parties and our own 
modules). This POM is split in 5 sub poms to organize them a bit (apache, 
netbeans, xxx, ...)

On this POM structure, we declare 3049 

  *   We have a parent pom. This one is used by all our 1628 modules. It 
defines the plugins versions and import the dependency management (explained 
above).
  *   We have 1628 modules which are using our parent pom (and then inherit the 
dependency management imported by the parent)
  *   We have an aggregator POMs structure which is responsible to include all 
the modules. Root aggregator is the one on which the full rebuild is triggered.



When Maven computes the reactor, it consumes a huge amount of memory (8 - 12 Gb)



I made a memory dump just before the end of reactor computation. I attached a 
screenshot in MNG-5661 as I 
feel it's related, please remove it if it's not appropriated.
On this screenshot we clearly see from where come from our memory issues: 
https://issues.apache.org/jira/secure/attachment/12955925/Maven-Reactor-Dump.png



The number of Dependency instance seems to be related to our structure. All our 
projects import our big dependency management (inherited from the parent) so we 
have 1628 x 3049 => ~ 5 000 000 instances.

This is also approximately the number of MavenProject, DefaultArtifact, 
VersionRange, etc... so I feel that the memory consumption comes from the fact 
that Maven doesn't detect such cases and duplicate everything.



Is Maven optimized to detect such cases? It would be great to ensure that 
Dependency (and others) instances are not duplicated.



Thanks,
Yann

Information in this e-mail and any attachments is confidential, and may not be 
copied or used by anyone other than the addressee, nor disclosed to any third 
party without our permission. There is no intention to create any legally 
binding contract or other binding commitment through the use of this electronic 
communication unless it is issued in accordance with the Experian Limited 
standard terms and conditions of purchase or other express written agreement 
between Experian Limited and the recipient. Although Experian has taken 
reasonable steps to ensure that this communication and any attachments are free 
from computer viruses, you are advised to take your own steps to ensure that 
they are actually virus free. 

Experian Ltd is authorised and regulated by the Financial Conduct Authority.
Companies Act information: Registered name: Experian Limited. Registered 
office: The Sir John Peace Building, Experian Way, NG2 Business Park, 
Nottingham, NG80 1ZZ, United Kingdom. Place of registration: England and Wales. 
Registered number: 653331.

Vos informations nominatives sont exploitées par la société Scorex S.A.M. 
(Groupe Experian)  dans le cadre du traitement ayant pour finalité "Gestion de 
la messagerie électronique professionnelle". Conformément à la loi n° 1.165, 
modifiée, vous disposez d'un droit d'accès, de rectification et de suppression 
en écrivant au Secrétariat Général de la société Scorex, sise, 2, rue de la 
Lüjerneta à Monaco (98000).

Your personal information is used by Scorex S.A.M. (Experian Group) as part of 
the processing for the purpose of "Professional e-mail management". In 
accordance with the law n ° 1.165, modified, you have a right of access, 
correction and suppression by writing to the General Secretariat of the 
company, located, 2, rue de la Lüjerneta in Monaco (98000).

Re: Maven Memory Consumption

[Karl Heinz Marbaise]

Hi Igor,

On 6/3/16 5:27 AM, Igor Fedorenko wrote:

I wonder if this fix also solves "remote shared parent" memory
inefficiency described in https://issues.apache.org/jira/browse/MNG-5669


So doing some research...

The DefaultProjectBuilder contains a method:

private ModelData readParent( Model childModel, ModelSource childSource, 
ModelBuildingRequest request,

  DefaultModelProblemCollector problems )

which is called for each module...

The information for the parent is cached ...So this looks to me it is 
fixed...but it is done before and is not related to MNG-6030...



Kind regards
Karl Heinz Marbaise

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Maven Memory Consumption

[Manfred Moser]

That sounds good to. Its awesome that this has been done and merged and will go 
out with the next release.

Manfred

Jason Dillon wrote on 2016-06-02 21:05:

> If its on by default I would expect folks to set
> MAVEN_OPTS=-Dmaven.logging=plain instead of magically making —batch do that.
> 
> If we mutate the cli api slightly to expose more details about the cli
> configuration to the Slf4jConfiguration then regular -Dmaven.logging=plain on
> command line would probably work too.  Right now the logging configuration has
> no context of the command-line params and can only use System.properties to
> fiddle with configuration.
> 
> If ^^^ was done then could also consider adding —color={yes|no} flag, though
> I felt odd hacking that in given that this is a pluggable aspect, and if you
> were using logbook backend it would be meaningless and potentially confusing.
> 
> —jason
> 
> 
> On June 2, 2016 at 8:37:18 PM, Manfred Moser (manf...@simpligility.com) wrote:
> 
> If we plan to switch it to on be default at a later stage we could
> automatically disable it in batch mode. And tell people to run in batch mode 
> on
> a CI server.  
> 
> Just a thought..  
> 
> Manfred  
> 
> Jason van Zyl wrote on 2016-06-02 19:52:  
> 
>> If the output comes out decently in color in CI consoles then it’s probably 
>> 
>> not an issue putting the color on by default. But I haven’t checked and  
>> suggested that the color be off by default to start with.  
>>  
>>> On Jun 2, 2016, at 5:15 PM, Hervé BOUTEMY  wrote:  
>>>  
>>> I merged the PR in the slf4j-gossip branch (and added a little improvement) 
>>>  
>>>  
>>> core ITs are ok (notice: ran without activating colors)  
>>> colors for Linux are not exactly the same as the screen dump: yellow from 
>>> the 
>>> 
>>> screen dump is bold white on Linux. This is ok for me  
>>>  
>>> Now, what's annoying is that:  
>>> - color is not enabled by default: I had to configure MAVEN_OPTS="-  
>>> Dmaven.logging=color"  
>>> - when redirecting content to file, color is not disabled automatically  
>>>  
>>> I don't know if this is a showstopper or not  
>>> I will continue to use it to see if there are unexpected side effects  
>>>  
>>> Regards,  
>>>  
>>> Hervé  
>>>  
>>> Le jeudi 2 juin 2016 09:21:46 Tamás Cservenák a écrit :  
 Olivier, if you refer to the slf4j-gossip branch, that IMHO Jason's PR  
 supersedes it.  
 Will drop that branch.  
  
 On Thu, Jun 2, 2016 at 8:38 AM Olivier Lamy  wrote:  
> well I think this color stuff has already been done differently but never 
>  
> accepted..  
>  
> On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:  
>> another feature that would be great for this release:  
>> https://github.com/apache/maven/pull/81  
>>  
>> I still didn't have time to work on it, but I like the screenshot  
>> The only thing that I'd like to check is: is tty detection working? ie  
>  
> does  
>  
>> color automatically disappear if there is no tty?  
>>  
>> Regards,  
>>  
>> Hervé  
>>  
>> Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :  
>>> +1  
>>> this is something that was often seen: this is great that it is fixed!  
>>>  
>>> For example, the last time I published Maven core site, the build  
>  
> simply  
>  
>>> failed because of PermgenSpace: now it is working like a charm...  
>>>  
>>> This release will be a must!  
>>>  
>>> Regards,  
>>>  
>>> Hervé  
>>>  
>>> Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :  
 Hi Manfred,  
  
 On 6/1/16 12:24 AM, Manfred Moser wrote:  
> I can feel your excitement coming through in the emails.. ;-)  
  
 Of course I'm excited ;-)  
  
 ...cause it's very important...I have had heard many customers  
 saying  
 they will not upgrade to newer versions of Maven exactly based on  
>  
> such  
>  
 issue(s)...  
  
 which is in general bad ...  
  
  
 This will break their argument ;-)...  
  
> Karl Heinz Marbaise wrote on 2016-05-31 15:14:  
>> Hi,  
>>  
>> tested without the patch (-Xmx6g) ...run time for the test  
>> project  
>>  
>> more  
>>  
>> than two 2 Minutes  
>>  
>> running with the patch (-Xmx1g):  
>>  
>> Run time ca. 27 seconds...  
>>  
>> also worked with -Xmx768m ...ca. 30 seconds...  
>>  
>> so looks very good...  
>>  
>> Let us wait what the IT's say...  
>>  
>> Kind regards  
>> Karl Heinz Marbaise  
>>  
>> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:  
>>> Hi,  
>>>  
>>> 

Re: Maven Memory Consumption

[Jason Dillon]

colors for Linux are not exactly the same as the screen dump: yellow from the 
screen dump is bold white on Linux. This is ok for me 
Terminals can render whatever colors they want for the ANSI colors and many 
have this configurable.

To be clear my terminal env has bold color rendering as yellow (very very very 
old preference I’ve had for a long time so much I forgot it was my preference).

We can certainly make things explicitly yellow though if that is preferred over 
simply bold.

—jason

Re: Maven Memory Consumption

[Jason Dillon]

If its on by default I would expect folks to set 
MAVEN_OPTS=-Dmaven.logging=plain instead of magically making —batch do that.

If we mutate the cli api slightly to expose more details about the cli 
configuration to the Slf4jConfiguration then regular -Dmaven.logging=plain on 
command line would probably work too.  Right now the logging configuration has 
no context of the command-line params and can only use System.properties to 
fiddle with configuration.

If ^^^ was done then could also consider adding —color={yes|no} flag, though I 
felt odd hacking that in given that this is a pluggable aspect, and if you were 
using logbook backend it would be meaningless and potentially confusing.

—jason


On June 2, 2016 at 8:37:18 PM, Manfred Moser (manf...@simpligility.com) wrote:

If we plan to switch it to on be default at a later stage we could 
automatically disable it in batch mode. And tell people to run in batch mode on 
a CI server.  

Just a thought..  

Manfred  

Jason van Zyl wrote on 2016-06-02 19:52:  

> If the output comes out decently in color in CI consoles then it’s probably  
> not an issue putting the color on by default. But I haven’t checked and  
> suggested that the color be off by default to start with.  
>  
>> On Jun 2, 2016, at 5:15 PM, Hervé BOUTEMY  wrote:  
>>  
>> I merged the PR in the slf4j-gossip branch (and added a little improvement)  
>>  
>> core ITs are ok (notice: ran without activating colors)  
>> colors for Linux are not exactly the same as the screen dump: yellow from 
>> the  
>> screen dump is bold white on Linux. This is ok for me  
>>  
>> Now, what's annoying is that:  
>> - color is not enabled by default: I had to configure MAVEN_OPTS="-  
>> Dmaven.logging=color"  
>> - when redirecting content to file, color is not disabled automatically  
>>  
>> I don't know if this is a showstopper or not  
>> I will continue to use it to see if there are unexpected side effects  
>>  
>> Regards,  
>>  
>> Hervé  
>>  
>> Le jeudi 2 juin 2016 09:21:46 Tamás Cservenák a écrit :  
>>> Olivier, if you refer to the slf4j-gossip branch, that IMHO Jason's PR  
>>> supersedes it.  
>>> Will drop that branch.  
>>>  
>>> On Thu, Jun 2, 2016 at 8:38 AM Olivier Lamy  wrote:  
 well I think this color stuff has already been done differently but never  
 accepted..  
  
 On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:  
> another feature that would be great for this release:  
> https://github.com/apache/maven/pull/81  
>  
> I still didn't have time to work on it, but I like the screenshot  
> The only thing that I'd like to check is: is tty detection working? ie  
  
 does  
  
> color automatically disappear if there is no tty?  
>  
> Regards,  
>  
> Hervé  
>  
> Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :  
>> +1  
>> this is something that was often seen: this is great that it is fixed!  
>>  
>> For example, the last time I published Maven core site, the build  
  
 simply  
  
>> failed because of PermgenSpace: now it is working like a charm...  
>>  
>> This release will be a must!  
>>  
>> Regards,  
>>  
>> Hervé  
>>  
>> Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :  
>>> Hi Manfred,  
>>>  
>>> On 6/1/16 12:24 AM, Manfred Moser wrote:  
 I can feel your excitement coming through in the emails.. ;-)  
>>>  
>>> Of course I'm excited ;-)  
>>>  
>>> ...cause it's very important...I have had heard many customers  
>>> saying  
>>> they will not upgrade to newer versions of Maven exactly based on  
  
 such  
  
>>> issue(s)...  
>>>  
>>> which is in general bad ...  
>>>  
>>>  
>>> This will break their argument ;-)...  
>>>  
 Karl Heinz Marbaise wrote on 2016-05-31 15:14:  
> Hi,  
>  
> tested without the patch (-Xmx6g) ...run time for the test  
> project  
>  
> more  
>  
> than two 2 Minutes  
>  
> running with the patch (-Xmx1g):  
>  
> Run time ca. 27 seconds...  
>  
> also worked with -Xmx768m ...ca. 30 seconds...  
>  
> so looks very good...  
>  
> Let us wait what the IT's say...  
>  
> Kind regards  
> Karl Heinz Marbaise  
>  
> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:  
>> Hi,  
>>  
>> after more investigation and an extremly good tip of  
  
 Andriy...(see  
  
>> MNG-6030) and in the end the solution:  
>>  
>> Using test project with 5000 modules just doing:  
>>  
>> mvn clean  
>>  
>> using the patch now in master  
>> 

Re: Maven Memory Consumption

[Jason Dillon]

On June 2, 2016 at 3:15:35 PM, Hervé BOUTEMY (herve.bout...@free.fr) wrote:
I merged the PR in the slf4j-gossip branch (and added a little improvement) 

core ITs are ok (notice: ran without activating colors) 
The testsuite certainly will not be very happy with color enabled as many tests 
are expecting exact strings in the output.  I had considered setting 
MAVEN_SKIP_RC env-var to avoid any local customization from polluting the 
testsuite execution.



colors for Linux are not exactly the same as the screen dump: yellow from the 
screen dump is bold white on Linux. This is ok for me 
Terminals can render whatever colors they want for the ANSI colors and many 
have this configurable.



Now, what's annoying is that: 
- color is not enabled by default: I had to configure MAVEN_OPTS="- 
Dmaven.logging=color" 
Seemed safer off by default then breaking the output for users where ansi is 
not supported and for whatever reason its not been detected as such to strip 
out.



- when redirecting content to file, color is not disabled automatically 
It may depend on how its redirected, my local testing shows that `mvn > foo’ 
disables ansi sequences, though I can not say I’ve done any exhaustive testing 
of this.

To be able to tell if System.out is an ANSI-aware terminal it may need a bit 
more love ala jline or more helpers in jansi to do more properly?

—jason

Re: Maven Memory Consumption

[Igor Fedorenko]

Somewhat related, Jason convinced me to opensource better logging
support for multithreaded builds I implemented some time ago.

https://github.com/takari/concurrent-build-logger

-- 
Regards,
Igor

On Thu, Jun 2, 2016, at 11:37 PM, Manfred Moser wrote:
> If we plan to switch it to on be default at a later stage we could
> automatically disable it in batch mode. And tell people to run in batch
> mode on a CI server. 
> 
> Just a thought.. 
> 
> Manfred
> 
> Jason van Zyl wrote on 2016-06-02 19:52:
> 
> > If the output comes out decently in color in CI consoles then it’s probably
> > not an issue putting the color on by default. But I haven’t checked and
> > suggested that the color be off by default to start with.
> > 
> >> On Jun 2, 2016, at 5:15 PM, Hervé BOUTEMY  wrote:
> >> 
> >> I merged the PR in the slf4j-gossip branch (and added a little improvement)
> >> 
> >> core ITs are ok (notice: ran without activating colors)
> >> colors for Linux are not exactly the same as the screen dump: yellow from 
> >> the 
> >> screen dump is bold white on Linux. This is ok for me
> >> 
> >> Now, what's annoying is that:
> >> - color is not enabled by default: I had to configure MAVEN_OPTS="-
> >> Dmaven.logging=color"
> >> - when redirecting content to file, color is not disabled automatically
> >> 
> >> I don't know if this is a showstopper or not
> >> I will continue to use it to see if there are unexpected side effects
> >> 
> >> Regards,
> >> 
> >> Hervé
> >> 
> >> Le jeudi 2 juin 2016 09:21:46 Tamás Cservenák a écrit :
> >>> Olivier, if you refer to the slf4j-gossip branch, that IMHO Jason's PR
> >>> supersedes it.
> >>> Will drop that branch.
> >>> 
> >>> On Thu, Jun 2, 2016 at 8:38 AM Olivier Lamy  wrote:
>  well I think this color stuff has already been done differently but never
>  accepted..
>  
>  On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:
> > another feature that would be great for this release:
> > https://github.com/apache/maven/pull/81
> > 
> > I still didn't have time to work on it, but I like the screenshot
> > The only thing that I'd like to check is: is tty detection working? ie
>  
>  does
>  
> > color automatically disappear if there is no tty?
> > 
> > Regards,
> > 
> > Hervé
> > 
> > Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :
> >> +1
> >> this is something that was often seen: this is great that it is fixed!
> >> 
> >> For example, the last time I published Maven core site, the build
>  
>  simply
>  
> >> failed because of PermgenSpace: now it is working like a charm...
> >> 
> >> This release will be a must!
> >> 
> >> Regards,
> >> 
> >> Hervé
> >> 
> >> Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
> >>> Hi Manfred,
> >>> 
> >>> On 6/1/16 12:24 AM, Manfred Moser wrote:
>  I can feel your excitement coming through in the emails.. ;-)
> >>> 
> >>> Of course I'm excited ;-)
> >>> 
> >>> ...cause it's very important...I have had heard many customers
> >>> saying
> >>> they will not upgrade to newer versions of Maven exactly based on
>  
>  such
>  
> >>> issue(s)...
> >>> 
> >>> which is in general bad ...
> >>> 
> >>> 
> >>> This will break their argument ;-)...
> >>> 
>  Karl Heinz Marbaise wrote on 2016-05-31 15:14:
> > Hi,
> > 
> > tested without the patch (-Xmx6g) ...run time for the test
> > project
> > 
> > more
> > 
> > than two 2 Minutes
> > 
> > running with the patch (-Xmx1g):
> > 
> > Run time ca. 27 seconds...
> > 
> > also worked with -Xmx768m ...ca. 30 seconds...
> > 
> > so looks very good...
> > 
> > Let us wait what the IT's say...
> > 
> > Kind regards
> > Karl Heinz Marbaise
> > 
> > On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> >> Hi,
> >> 
> >> after more investigation and an extremly good tip of
>  
>  Andriy...(see
>  
> >> MNG-6030) and in the end the solution:
> >> 
> >> Using test project with 5000 modules just doing:
> >> 
> >> mvn clean
> >> 
> >> using the patch now in master
> >> (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> >> extremely reduces the memory footprint...
> > 
> > https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > 
> >> /M
> >> aven340-with-patch-5000.png
> >> 
> >> 
> >> This shows the result using the patch
> > 
> >> The following shows Maven 3.3.9:
> > https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > 
> >> /M

Re: Maven Memory Consumption

[Manfred Moser]

If we plan to switch it to on be default at a later stage we could 
automatically disable it in batch mode. And tell people to run in batch mode on 
a CI server. 

Just a thought.. 

Manfred

Jason van Zyl wrote on 2016-06-02 19:52:

> If the output comes out decently in color in CI consoles then it’s probably
> not an issue putting the color on by default. But I haven’t checked and
> suggested that the color be off by default to start with.
> 
>> On Jun 2, 2016, at 5:15 PM, Hervé BOUTEMY  wrote:
>> 
>> I merged the PR in the slf4j-gossip branch (and added a little improvement)
>> 
>> core ITs are ok (notice: ran without activating colors)
>> colors for Linux are not exactly the same as the screen dump: yellow from 
>> the 
>> screen dump is bold white on Linux. This is ok for me
>> 
>> Now, what's annoying is that:
>> - color is not enabled by default: I had to configure MAVEN_OPTS="-
>> Dmaven.logging=color"
>> - when redirecting content to file, color is not disabled automatically
>> 
>> I don't know if this is a showstopper or not
>> I will continue to use it to see if there are unexpected side effects
>> 
>> Regards,
>> 
>> Hervé
>> 
>> Le jeudi 2 juin 2016 09:21:46 Tamás Cservenák a écrit :
>>> Olivier, if you refer to the slf4j-gossip branch, that IMHO Jason's PR
>>> supersedes it.
>>> Will drop that branch.
>>> 
>>> On Thu, Jun 2, 2016 at 8:38 AM Olivier Lamy  wrote:
 well I think this color stuff has already been done differently but never
 accepted..
 
 On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:
> another feature that would be great for this release:
> https://github.com/apache/maven/pull/81
> 
> I still didn't have time to work on it, but I like the screenshot
> The only thing that I'd like to check is: is tty detection working? ie
 
 does
 
> color automatically disappear if there is no tty?
> 
> Regards,
> 
> Hervé
> 
> Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :
>> +1
>> this is something that was often seen: this is great that it is fixed!
>> 
>> For example, the last time I published Maven core site, the build
 
 simply
 
>> failed because of PermgenSpace: now it is working like a charm...
>> 
>> This release will be a must!
>> 
>> Regards,
>> 
>> Hervé
>> 
>> Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
>>> Hi Manfred,
>>> 
>>> On 6/1/16 12:24 AM, Manfred Moser wrote:
 I can feel your excitement coming through in the emails.. ;-)
>>> 
>>> Of course I'm excited ;-)
>>> 
>>> ...cause it's very important...I have had heard many customers
>>> saying
>>> they will not upgrade to newer versions of Maven exactly based on
 
 such
 
>>> issue(s)...
>>> 
>>> which is in general bad ...
>>> 
>>> 
>>> This will break their argument ;-)...
>>> 
 Karl Heinz Marbaise wrote on 2016-05-31 15:14:
> Hi,
> 
> tested without the patch (-Xmx6g) ...run time for the test
> project
> 
> more
> 
> than two 2 Minutes
> 
> running with the patch (-Xmx1g):
> 
> Run time ca. 27 seconds...
> 
> also worked with -Xmx768m ...ca. 30 seconds...
> 
> so looks very good...
> 
> Let us wait what the IT's say...
> 
> Kind regards
> Karl Heinz Marbaise
> 
> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
>> Hi,
>> 
>> after more investigation and an extremly good tip of
 
 Andriy...(see
 
>> MNG-6030) and in the end the solution:
>> 
>> Using test project with 5000 modules just doing:
>> 
>> mvn clean
>> 
>> using the patch now in master
>> (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
>> extremely reduces the memory footprint...
> 
> https://github.com/khmarbaise/maven-test-project-generator/blob/master
> 
>> /M
>> aven340-with-patch-5000.png
>> 
>> 
>> This shows the result using the patch
> 
>> The following shows Maven 3.3.9:
> https://github.com/khmarbaise/maven-test-project-generator/blob/master
> 
>> /M
>> aven339-5000.png
>> 
>> 
>> Many thanks to Andriy for the support and help...
>> 
>> we will see if not IT's will fail on the change.
>> 
>> 
>> Kind regards
>> Karl Heinz Marbaise
>> 
>> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
>>> Hi,
>>> 
>>> i started a little bit more detailed analysis..
>>> 
>>> very simple via JConsole and running the different versions...
>>> 
>>> I have 

Re: Maven Memory Consumption

[Igor Fedorenko]

I wonder if this fix also solves "remote shared parent" memory
inefficiency described in https://issues.apache.org/jira/browse/MNG-5669

-- 
Regards,
Igor

On Tue, May 31, 2016, at 06:14 PM, Karl Heinz Marbaise wrote:
> Hi,
> 
> tested without the patch (-Xmx6g) ...run time for the test project more 
> than two 2 Minutes
> 
> running with the patch (-Xmx1g):
> 
> Run time ca. 27 seconds...
> 
> also worked with -Xmx768m ...ca. 30 seconds...
> 
> so looks very good...
> 
> Let us wait what the IT's say...
> 
> Kind regards
> Karl Heinz Marbaise
> 
> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> > Hi,
> >
> > after more investigation and an extremly good tip of Andriy...(see
> > MNG-6030) and in the end the solution:
> >
> > Using test project with 5000 modules just doing:
> >
> > mvn clean
> >
> > using the patch now in master (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> > extremely reduces the memory footprint...
> >
> > https://github.com/khmarbaise/maven-test-project-generator/blob/master/Maven340-with-patch-5000.png
> >
> >
> > This shows the result using the patch
> >
> > The following shows Maven 3.3.9:
> >
> > https://github.com/khmarbaise/maven-test-project-generator/blob/master/Maven339-5000.png
> >
> >
> > Many thanks to Andriy for the support and help...
> >
> > we will see if not IT's will fail on the change.
> >
> >
> > Kind regards
> > Karl Heinz Marbaise
> >
> > On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
> >> Hi,
> >>
> >> i started a little bit more detailed analysis..
> >>
> >> very simple via JConsole and running the different versions...
> >>
> >> I have summarized this here:
> >>
> >> https://github.com/khmarbaise/maven-test-project-generator
> >>
> >> Kind regards
> >> Karl Heinz Marbaise
> >>
> >> On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
> >>> Hi to all,
> >>>
> >>> i have a question concerning the memory consumption...
> >>>
> >>> If i run maven with the same JDK and the same reactor and build with the
> >>> same parameter and plugins...
> >>>
> >>> will the printout at the end of the build (Final Memory) something
> >>> realiable about the consumption of the JVM during the build ?...Or is it
> >>> at least a hint...or would i need to do something different (BTW:
> >>> Someone has a hint about that?) ...
> >>>
> >>>
> >>> [INFO] BUILD SUCCESS
> >>> [INFO]
> >>> 
> >>> [INFO] Total time: 6.431 s
> >>> [INFO] Finished at: 2016-04-17T17:46:58+02:00
> >>> [INFO] Final Memory: 47M/638M
> >>>
> >>> So if i ran the same build with different Maven versions so could this
> >>> give us a hint where more memory is consumed ...(to identify where and
> >>> why is a different story)...
> >>>
> >>> Kind regards
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
> 

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Maven Memory Consumption

[Jason van Zyl]

If the output comes out decently in color in CI consoles then it’s probably not 
an issue putting the color on by default. But I haven’t checked and suggested 
that the color be off by default to start with.

> On Jun 2, 2016, at 5:15 PM, Hervé BOUTEMY  wrote:
> 
> I merged the PR in the slf4j-gossip branch (and added a little improvement)
> 
> core ITs are ok (notice: ran without activating colors)
> colors for Linux are not exactly the same as the screen dump: yellow from the 
> screen dump is bold white on Linux. This is ok for me
> 
> Now, what's annoying is that:
> - color is not enabled by default: I had to configure MAVEN_OPTS="-
> Dmaven.logging=color"
> - when redirecting content to file, color is not disabled automatically
> 
> I don't know if this is a showstopper or not
> I will continue to use it to see if there are unexpected side effects
> 
> Regards,
> 
> Hervé
> 
> Le jeudi 2 juin 2016 09:21:46 Tamás Cservenák a écrit :
>> Olivier, if you refer to the slf4j-gossip branch, that IMHO Jason's PR
>> supersedes it.
>> Will drop that branch.
>> 
>> On Thu, Jun 2, 2016 at 8:38 AM Olivier Lamy  wrote:
>>> well I think this color stuff has already been done differently but never
>>> accepted..
>>> 
>>> On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:
 another feature that would be great for this release:
 https://github.com/apache/maven/pull/81
 
 I still didn't have time to work on it, but I like the screenshot
 The only thing that I'd like to check is: is tty detection working? ie
>>> 
>>> does
>>> 
 color automatically disappear if there is no tty?
 
 Regards,
 
 Hervé
 
 Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :
> +1
> this is something that was often seen: this is great that it is fixed!
> 
> For example, the last time I published Maven core site, the build
>>> 
>>> simply
>>> 
> failed because of PermgenSpace: now it is working like a charm...
> 
> This release will be a must!
> 
> Regards,
> 
> Hervé
> 
> Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
>> Hi Manfred,
>> 
>> On 6/1/16 12:24 AM, Manfred Moser wrote:
>>> I can feel your excitement coming through in the emails.. ;-)
>> 
>> Of course I'm excited ;-)
>> 
>> ...cause it's very important...I have had heard many customers
>> saying
>> they will not upgrade to newer versions of Maven exactly based on
>>> 
>>> such
>>> 
>> issue(s)...
>> 
>> which is in general bad ...
>> 
>> 
>> This will break their argument ;-)...
>> 
>>> Karl Heinz Marbaise wrote on 2016-05-31 15:14:
 Hi,
 
 tested without the patch (-Xmx6g) ...run time for the test
 project
 
 more
 
 than two 2 Minutes
 
 running with the patch (-Xmx1g):
 
 Run time ca. 27 seconds...
 
 also worked with -Xmx768m ...ca. 30 seconds...
 
 so looks very good...
 
 Let us wait what the IT's say...
 
 Kind regards
 Karl Heinz Marbaise
 
 On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> Hi,
> 
> after more investigation and an extremly good tip of
>>> 
>>> Andriy...(see
>>> 
> MNG-6030) and in the end the solution:
> 
> Using test project with 5000 modules just doing:
> 
> mvn clean
> 
> using the patch now in master
> (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> extremely reduces the memory footprint...
 
 https://github.com/khmarbaise/maven-test-project-generator/blob/master
 
> /M
> aven340-with-patch-5000.png
> 
> 
> This shows the result using the patch
 
> The following shows Maven 3.3.9:
 https://github.com/khmarbaise/maven-test-project-generator/blob/master
 
> /M
> aven339-5000.png
> 
> 
> Many thanks to Andriy for the support and help...
> 
> we will see if not IT's will fail on the change.
> 
> 
> Kind regards
> Karl Heinz Marbaise
> 
> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
>> Hi,
>> 
>> i started a little bit more detailed analysis..
>> 
>> very simple via JConsole and running the different versions...
>> 
>> I have summarized this here:
>> 
>> https://github.com/khmarbaise/maven-test-project-generator
>> 
>> Kind regards
>> Karl Heinz Marbaise
>> 
>> On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
>>> Hi to all,
>>> 
>>> i have a question concerning the memory consumption...
>>> 
>>> If i run maven with the same JDK 

Re: Maven Memory Consumption

[Hervé BOUTEMY]

I merged the PR in the slf4j-gossip branch (and added a little improvement)

core ITs are ok (notice: ran without activating colors)
colors for Linux are not exactly the same as the screen dump: yellow from the 
screen dump is bold white on Linux. This is ok for me

Now, what's annoying is that:
- color is not enabled by default: I had to configure MAVEN_OPTS="-
Dmaven.logging=color"
- when redirecting content to file, color is not disabled automatically

I don't know if this is a showstopper or not
I will continue to use it to see if there are unexpected side effects

Regards,

Hervé

Le jeudi 2 juin 2016 09:21:46 Tamás Cservenák a écrit :
> Olivier, if you refer to the slf4j-gossip branch, that IMHO Jason's PR
> supersedes it.
> Will drop that branch.
> 
> On Thu, Jun 2, 2016 at 8:38 AM Olivier Lamy  wrote:
> > well I think this color stuff has already been done differently but never
> > accepted..
> > 
> > On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:
> > > another feature that would be great for this release:
> > > https://github.com/apache/maven/pull/81
> > > 
> > > I still didn't have time to work on it, but I like the screenshot
> > > The only thing that I'd like to check is: is tty detection working? ie
> > 
> > does
> > 
> > > color automatically disappear if there is no tty?
> > > 
> > > Regards,
> > > 
> > > Hervé
> > > 
> > > Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :
> > > > +1
> > > > this is something that was often seen: this is great that it is fixed!
> > > > 
> > > > For example, the last time I published Maven core site, the build
> > 
> > simply
> > 
> > > > failed because of PermgenSpace: now it is working like a charm...
> > > > 
> > > > This release will be a must!
> > > > 
> > > > Regards,
> > > > 
> > > > Hervé
> > > > 
> > > > Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
> > > > > Hi Manfred,
> > > > > 
> > > > > On 6/1/16 12:24 AM, Manfred Moser wrote:
> > > > > > I can feel your excitement coming through in the emails.. ;-)
> > > > > 
> > > > > Of course I'm excited ;-)
> > > > > 
> > > > > ...cause it's very important...I have had heard many customers
> > > > > saying
> > > > > they will not upgrade to newer versions of Maven exactly based on
> > 
> > such
> > 
> > > > > issue(s)...
> > > > > 
> > > > > which is in general bad ...
> > > > > 
> > > > > 
> > > > > This will break their argument ;-)...
> > > > > 
> > > > > > Karl Heinz Marbaise wrote on 2016-05-31 15:14:
> > > > > >> Hi,
> > > > > >> 
> > > > > >> tested without the patch (-Xmx6g) ...run time for the test
> > > > > >> project
> > > 
> > > more
> > > 
> > > > > >> than two 2 Minutes
> > > > > >> 
> > > > > >> running with the patch (-Xmx1g):
> > > > > >> 
> > > > > >> Run time ca. 27 seconds...
> > > > > >> 
> > > > > >> also worked with -Xmx768m ...ca. 30 seconds...
> > > > > >> 
> > > > > >> so looks very good...
> > > > > >> 
> > > > > >> Let us wait what the IT's say...
> > > > > >> 
> > > > > >> Kind regards
> > > > > >> Karl Heinz Marbaise
> > > > > >> 
> > > > > >> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> > > > > >>> Hi,
> > > > > >>> 
> > > > > >>> after more investigation and an extremly good tip of
> > 
> > Andriy...(see
> > 
> > > > > >>> MNG-6030) and in the end the solution:
> > > > > >>> 
> > > > > >>> Using test project with 5000 modules just doing:
> > > > > >>> 
> > > > > >>> mvn clean
> > > > > >>> 
> > > > > >>> using the patch now in master
> > > > > >>> (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> > > > > >>> extremely reduces the memory footprint...
> > > 
> > > https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > > 
> > > > > >>> /M
> > > > > >>> aven340-with-patch-5000.png
> > > > > >>> 
> > > > > >>> 
> > > > > >>> This shows the result using the patch
> > > 
> > > > > >>> The following shows Maven 3.3.9:
> > > https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > > 
> > > > > >>> /M
> > > > > >>> aven339-5000.png
> > > > > >>> 
> > > > > >>> 
> > > > > >>> Many thanks to Andriy for the support and help...
> > > > > >>> 
> > > > > >>> we will see if not IT's will fail on the change.
> > > > > >>> 
> > > > > >>> 
> > > > > >>> Kind regards
> > > > > >>> Karl Heinz Marbaise
> > > > > >>> 
> > > > > >>> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
> > > > >  Hi,
> > > > >  
> > > > >  i started a little bit more detailed analysis..
> > > > >  
> > > > >  very simple via JConsole and running the different versions...
> > > > >  
> > > > >  I have summarized this here:
> > > > >  
> > > > >  https://github.com/khmarbaise/maven-test-project-generator
> > > > >  
> > > > >  Kind regards
> > > > >  Karl Heinz Marbaise
> > > > >  
> > > > >  On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
> > > > > > Hi to all,
> > > > > > 
> > > > > > i have a question concerning the memory consumption...
> > > > > 

Re: Maven Memory Consumption

[Tamás Cservenák]

Olivier, if you refer to the slf4j-gossip branch, that IMHO Jason's PR
supersedes it.
Will drop that branch.

On Thu, Jun 2, 2016 at 8:38 AM Olivier Lamy  wrote:

> well I think this color stuff has already been done differently but never
> accepted..
>
> On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:
>
> > another feature that would be great for this release:
> > https://github.com/apache/maven/pull/81
> >
> > I still didn't have time to work on it, but I like the screenshot
> > The only thing that I'd like to check is: is tty detection working? ie
> does
> > color automatically disappear if there is no tty?
> >
> > Regards,
> >
> > Hervé
> >
> > Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :
> > > +1
> > > this is something that was often seen: this is great that it is fixed!
> > >
> > > For example, the last time I published Maven core site, the build
> simply
> > > failed because of PermgenSpace: now it is working like a charm...
> > >
> > > This release will be a must!
> > >
> > > Regards,
> > >
> > > Hervé
> > >
> > > Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
> > > > Hi Manfred,
> > > >
> > > > On 6/1/16 12:24 AM, Manfred Moser wrote:
> > > > > I can feel your excitement coming through in the emails.. ;-)
> > > >
> > > > Of course I'm excited ;-)
> > > >
> > > > ...cause it's very important...I have had heard many customers saying
> > > > they will not upgrade to newer versions of Maven exactly based on
> such
> > > > issue(s)...
> > > >
> > > > which is in general bad ...
> > > >
> > > >
> > > > This will break their argument ;-)...
> > > >
> > > > > Karl Heinz Marbaise wrote on 2016-05-31 15:14:
> > > > >> Hi,
> > > > >>
> > > > >> tested without the patch (-Xmx6g) ...run time for the test project
> > more
> > > > >> than two 2 Minutes
> > > > >>
> > > > >> running with the patch (-Xmx1g):
> > > > >>
> > > > >> Run time ca. 27 seconds...
> > > > >>
> > > > >> also worked with -Xmx768m ...ca. 30 seconds...
> > > > >>
> > > > >> so looks very good...
> > > > >>
> > > > >> Let us wait what the IT's say...
> > > > >>
> > > > >> Kind regards
> > > > >> Karl Heinz Marbaise
> > > > >>
> > > > >> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> > > > >>> Hi,
> > > > >>>
> > > > >>> after more investigation and an extremly good tip of
> Andriy...(see
> > > > >>> MNG-6030) and in the end the solution:
> > > > >>>
> > > > >>> Using test project with 5000 modules just doing:
> > > > >>>
> > > > >>> mvn clean
> > > > >>>
> > > > >>> using the patch now in master
> > > > >>> (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> > > > >>> extremely reduces the memory footprint...
> > > > >>>
> > > > >>>
> > https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > > > >>> /M
> > > > >>> aven340-with-patch-5000.png
> > > > >>>
> > > > >>>
> > > > >>> This shows the result using the patch
> > > > >>>
> > > > >>> The following shows Maven 3.3.9:
> > > > >>>
> > > > >>>
> > https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > > > >>> /M
> > > > >>> aven339-5000.png
> > > > >>>
> > > > >>>
> > > > >>> Many thanks to Andriy for the support and help...
> > > > >>>
> > > > >>> we will see if not IT's will fail on the change.
> > > > >>>
> > > > >>>
> > > > >>> Kind regards
> > > > >>> Karl Heinz Marbaise
> > > > >>>
> > > > >>> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
> > > >  Hi,
> > > > 
> > > >  i started a little bit more detailed analysis..
> > > > 
> > > >  very simple via JConsole and running the different versions...
> > > > 
> > > >  I have summarized this here:
> > > > 
> > > >  https://github.com/khmarbaise/maven-test-project-generator
> > > > 
> > > >  Kind regards
> > > >  Karl Heinz Marbaise
> > > > 
> > > >  On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
> > > > > Hi to all,
> > > > >
> > > > > i have a question concerning the memory consumption...
> > > > >
> > > > > If i run maven with the same JDK and the same reactor and build
> > with
> > > > > the
> > > > > same parameter and plugins...
> > > > >
> > > > > will the printout at the end of the build (Final Memory)
> > something
> > > > > realiable about the consumption of the JVM during the build
> > ?...Or
> > > > > is
> > > > > it
> > > > > at least a hint...or would i need to do something different
> (BTW:
> > > > > Someone has a hint about that?) ...
> > > > >
> > > > >
> > > > > [INFO] BUILD SUCCESS
> > > > > [INFO]
> > > > >
> > 
> > > > > --
> > > > > --
> > > > > [INFO] Total time: 6.431 s
> > > > > [INFO] Finished at: 2016-04-17T17:46:58+02:00
> > > > > [INFO] Final Memory: 47M/638M
> > > > >
> > > > > So if i ran the same build with different Maven versions so
> could
> > > > > this
> > > > > give 

Re: Maven Memory Consumption

[Olivier Lamy]

well I think this color stuff has already been done differently but never
accepted..

On 2 June 2016 at 16:28, Hervé BOUTEMY  wrote:

> another feature that would be great for this release:
> https://github.com/apache/maven/pull/81
>
> I still didn't have time to work on it, but I like the screenshot
> The only thing that I'd like to check is: is tty detection working? ie does
> color automatically disappear if there is no tty?
>
> Regards,
>
> Hervé
>
> Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :
> > +1
> > this is something that was often seen: this is great that it is fixed!
> >
> > For example, the last time I published Maven core site, the build simply
> > failed because of PermgenSpace: now it is working like a charm...
> >
> > This release will be a must!
> >
> > Regards,
> >
> > Hervé
> >
> > Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
> > > Hi Manfred,
> > >
> > > On 6/1/16 12:24 AM, Manfred Moser wrote:
> > > > I can feel your excitement coming through in the emails.. ;-)
> > >
> > > Of course I'm excited ;-)
> > >
> > > ...cause it's very important...I have had heard many customers saying
> > > they will not upgrade to newer versions of Maven exactly based on such
> > > issue(s)...
> > >
> > > which is in general bad ...
> > >
> > >
> > > This will break their argument ;-)...
> > >
> > > > Karl Heinz Marbaise wrote on 2016-05-31 15:14:
> > > >> Hi,
> > > >>
> > > >> tested without the patch (-Xmx6g) ...run time for the test project
> more
> > > >> than two 2 Minutes
> > > >>
> > > >> running with the patch (-Xmx1g):
> > > >>
> > > >> Run time ca. 27 seconds...
> > > >>
> > > >> also worked with -Xmx768m ...ca. 30 seconds...
> > > >>
> > > >> so looks very good...
> > > >>
> > > >> Let us wait what the IT's say...
> > > >>
> > > >> Kind regards
> > > >> Karl Heinz Marbaise
> > > >>
> > > >> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> > > >>> Hi,
> > > >>>
> > > >>> after more investigation and an extremly good tip of Andriy...(see
> > > >>> MNG-6030) and in the end the solution:
> > > >>>
> > > >>> Using test project with 5000 modules just doing:
> > > >>>
> > > >>> mvn clean
> > > >>>
> > > >>> using the patch now in master
> > > >>> (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> > > >>> extremely reduces the memory footprint...
> > > >>>
> > > >>>
> https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > > >>> /M
> > > >>> aven340-with-patch-5000.png
> > > >>>
> > > >>>
> > > >>> This shows the result using the patch
> > > >>>
> > > >>> The following shows Maven 3.3.9:
> > > >>>
> > > >>>
> https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > > >>> /M
> > > >>> aven339-5000.png
> > > >>>
> > > >>>
> > > >>> Many thanks to Andriy for the support and help...
> > > >>>
> > > >>> we will see if not IT's will fail on the change.
> > > >>>
> > > >>>
> > > >>> Kind regards
> > > >>> Karl Heinz Marbaise
> > > >>>
> > > >>> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
> > >  Hi,
> > > 
> > >  i started a little bit more detailed analysis..
> > > 
> > >  very simple via JConsole and running the different versions...
> > > 
> > >  I have summarized this here:
> > > 
> > >  https://github.com/khmarbaise/maven-test-project-generator
> > > 
> > >  Kind regards
> > >  Karl Heinz Marbaise
> > > 
> > >  On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
> > > > Hi to all,
> > > >
> > > > i have a question concerning the memory consumption...
> > > >
> > > > If i run maven with the same JDK and the same reactor and build
> with
> > > > the
> > > > same parameter and plugins...
> > > >
> > > > will the printout at the end of the build (Final Memory)
> something
> > > > realiable about the consumption of the JVM during the build
> ?...Or
> > > > is
> > > > it
> > > > at least a hint...or would i need to do something different (BTW:
> > > > Someone has a hint about that?) ...
> > > >
> > > >
> > > > [INFO] BUILD SUCCESS
> > > > [INFO]
> > > >
> 
> > > > --
> > > > --
> > > > [INFO] Total time: 6.431 s
> > > > [INFO] Finished at: 2016-04-17T17:46:58+02:00
> > > > [INFO] Final Memory: 47M/638M
> > > >
> > > > So if i ran the same build with different Maven versions so could
> > > > this
> > > > give us a hint where more memory is consumed ...(to identify
> where
> > > > and
> > > > why is a different story)...
> > > >
> > > > Kind regards
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > > For additional commands, e-mail: dev-h...@maven.apache.org
> >
> > -
> > To unsubscribe, e-mail: 

Re: Maven Memory Consumption

[Hervé BOUTEMY]

another feature that would be great for this release:
https://github.com/apache/maven/pull/81

I still didn't have time to work on it, but I like the screenshot
The only thing that I'd like to check is: is tty detection working? ie does 
color automatically disappear if there is no tty?

Regards,

Hervé

Le jeudi 2 juin 2016 08:23:57 Hervé BOUTEMY a écrit :
> +1
> this is something that was often seen: this is great that it is fixed!
> 
> For example, the last time I published Maven core site, the build simply
> failed because of PermgenSpace: now it is working like a charm...
> 
> This release will be a must!
> 
> Regards,
> 
> Hervé
> 
> Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
> > Hi Manfred,
> > 
> > On 6/1/16 12:24 AM, Manfred Moser wrote:
> > > I can feel your excitement coming through in the emails.. ;-)
> > 
> > Of course I'm excited ;-)
> > 
> > ...cause it's very important...I have had heard many customers saying
> > they will not upgrade to newer versions of Maven exactly based on such
> > issue(s)...
> > 
> > which is in general bad ...
> > 
> > 
> > This will break their argument ;-)...
> > 
> > > Karl Heinz Marbaise wrote on 2016-05-31 15:14:
> > >> Hi,
> > >> 
> > >> tested without the patch (-Xmx6g) ...run time for the test project more
> > >> than two 2 Minutes
> > >> 
> > >> running with the patch (-Xmx1g):
> > >> 
> > >> Run time ca. 27 seconds...
> > >> 
> > >> also worked with -Xmx768m ...ca. 30 seconds...
> > >> 
> > >> so looks very good...
> > >> 
> > >> Let us wait what the IT's say...
> > >> 
> > >> Kind regards
> > >> Karl Heinz Marbaise
> > >> 
> > >> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> > >>> Hi,
> > >>> 
> > >>> after more investigation and an extremly good tip of Andriy...(see
> > >>> MNG-6030) and in the end the solution:
> > >>> 
> > >>> Using test project with 5000 modules just doing:
> > >>> 
> > >>> mvn clean
> > >>> 
> > >>> using the patch now in master
> > >>> (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> > >>> extremely reduces the memory footprint...
> > >>> 
> > >>> https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > >>> /M
> > >>> aven340-with-patch-5000.png
> > >>> 
> > >>> 
> > >>> This shows the result using the patch
> > >>> 
> > >>> The following shows Maven 3.3.9:
> > >>> 
> > >>> https://github.com/khmarbaise/maven-test-project-generator/blob/master
> > >>> /M
> > >>> aven339-5000.png
> > >>> 
> > >>> 
> > >>> Many thanks to Andriy for the support and help...
> > >>> 
> > >>> we will see if not IT's will fail on the change.
> > >>> 
> > >>> 
> > >>> Kind regards
> > >>> Karl Heinz Marbaise
> > >>> 
> > >>> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
> >  Hi,
> >  
> >  i started a little bit more detailed analysis..
> >  
> >  very simple via JConsole and running the different versions...
> >  
> >  I have summarized this here:
> >  
> >  https://github.com/khmarbaise/maven-test-project-generator
> >  
> >  Kind regards
> >  Karl Heinz Marbaise
> >  
> >  On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
> > > Hi to all,
> > > 
> > > i have a question concerning the memory consumption...
> > > 
> > > If i run maven with the same JDK and the same reactor and build with
> > > the
> > > same parameter and plugins...
> > > 
> > > will the printout at the end of the build (Final Memory) something
> > > realiable about the consumption of the JVM during the build ?...Or
> > > is
> > > it
> > > at least a hint...or would i need to do something different (BTW:
> > > Someone has a hint about that?) ...
> > > 
> > > 
> > > [INFO] BUILD SUCCESS
> > > [INFO]
> > > 
> > > --
> > > --
> > > [INFO] Total time: 6.431 s
> > > [INFO] Finished at: 2016-04-17T17:46:58+02:00
> > > [INFO] Final Memory: 47M/638M
> > > 
> > > So if i ran the same build with different Maven versions so could
> > > this
> > > give us a hint where more memory is consumed ...(to identify where
> > > and
> > > why is a different story)...
> > > 
> > > Kind regards
> > 
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > For additional commands, e-mail: dev-h...@maven.apache.org
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Maven Memory Consumption

[Hervé BOUTEMY]

+1
this is something that was often seen: this is great that it is fixed!

For example, the last time I published Maven core site, the build simply 
failed because of PermgenSpace: now it is working like a charm...

This release will be a must!

Regards,

Hervé

Le mercredi 1 juin 2016 20:12:33 Karl Heinz Marbaise a écrit :
> Hi Manfred,
> 
> On 6/1/16 12:24 AM, Manfred Moser wrote:
> > I can feel your excitement coming through in the emails.. ;-)
> 
> Of course I'm excited ;-)
> 
> ...cause it's very important...I have had heard many customers saying
> they will not upgrade to newer versions of Maven exactly based on such
> issue(s)...
> 
> which is in general bad ...
> 
> 
> This will break their argument ;-)...
> 
> > Karl Heinz Marbaise wrote on 2016-05-31 15:14:
> >> Hi,
> >> 
> >> tested without the patch (-Xmx6g) ...run time for the test project more
> >> than two 2 Minutes
> >> 
> >> running with the patch (-Xmx1g):
> >> 
> >> Run time ca. 27 seconds...
> >> 
> >> also worked with -Xmx768m ...ca. 30 seconds...
> >> 
> >> so looks very good...
> >> 
> >> Let us wait what the IT's say...
> >> 
> >> Kind regards
> >> Karl Heinz Marbaise
> >> 
> >> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
> >>> Hi,
> >>> 
> >>> after more investigation and an extremly good tip of Andriy...(see
> >>> MNG-6030) and in the end the solution:
> >>> 
> >>> Using test project with 5000 modules just doing:
> >>> 
> >>> mvn clean
> >>> 
> >>> using the patch now in master (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
> >>> extremely reduces the memory footprint...
> >>> 
> >>> https://github.com/khmarbaise/maven-test-project-generator/blob/master/M
> >>> aven340-with-patch-5000.png
> >>> 
> >>> 
> >>> This shows the result using the patch
> >>> 
> >>> The following shows Maven 3.3.9:
> >>> 
> >>> https://github.com/khmarbaise/maven-test-project-generator/blob/master/M
> >>> aven339-5000.png
> >>> 
> >>> 
> >>> Many thanks to Andriy for the support and help...
> >>> 
> >>> we will see if not IT's will fail on the change.
> >>> 
> >>> 
> >>> Kind regards
> >>> Karl Heinz Marbaise
> >>> 
> >>> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
>  Hi,
>  
>  i started a little bit more detailed analysis..
>  
>  very simple via JConsole and running the different versions...
>  
>  I have summarized this here:
>  
>  https://github.com/khmarbaise/maven-test-project-generator
>  
>  Kind regards
>  Karl Heinz Marbaise
>  
>  On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
> > Hi to all,
> > 
> > i have a question concerning the memory consumption...
> > 
> > If i run maven with the same JDK and the same reactor and build with
> > the
> > same parameter and plugins...
> > 
> > will the printout at the end of the build (Final Memory) something
> > realiable about the consumption of the JVM during the build ?...Or is
> > it
> > at least a hint...or would i need to do something different (BTW:
> > Someone has a hint about that?) ...
> > 
> > 
> > [INFO] BUILD SUCCESS
> > [INFO]
> > --
> > --
> > [INFO] Total time: 6.431 s
> > [INFO] Finished at: 2016-04-17T17:46:58+02:00
> > [INFO] Final Memory: 47M/638M
> > 
> > So if i ran the same build with different Maven versions so could this
> > give us a hint where more memory is consumed ...(to identify where and
> > why is a different story)...
> > 
> > Kind regards
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Maven Memory Consumption

[Karl Heinz Marbaise]

Hi Manfred,

On 6/1/16 12:24 AM, Manfred Moser wrote:

I can feel your excitement coming through in the emails.. ;-)


Of course I'm excited ;-)

...cause it's very important...I have had heard many customers saying 
they will not upgrade to newer versions of Maven exactly based on such 
issue(s)...


which is in general bad ...


This will break their argument ;-)...




Karl Heinz Marbaise wrote on 2016-05-31 15:14:


Hi,

tested without the patch (-Xmx6g) ...run time for the test project more
than two 2 Minutes

running with the patch (-Xmx1g):

Run time ca. 27 seconds...

also worked with -Xmx768m ...ca. 30 seconds...

so looks very good...

Let us wait what the IT's say...

Kind regards
Karl Heinz Marbaise

On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:

Hi,

after more investigation and an extremly good tip of Andriy...(see
MNG-6030) and in the end the solution:

Using test project with 5000 modules just doing:

mvn clean

using the patch now in master (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
extremely reduces the memory footprint...

https://github.com/khmarbaise/maven-test-project-generator/blob/master/Maven340-with-patch-5000.png


This shows the result using the patch

The following shows Maven 3.3.9:

https://github.com/khmarbaise/maven-test-project-generator/blob/master/Maven339-5000.png


Many thanks to Andriy for the support and help...

we will see if not IT's will fail on the change.


Kind regards
Karl Heinz Marbaise

On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:

Hi,

i started a little bit more detailed analysis..

very simple via JConsole and running the different versions...

I have summarized this here:

https://github.com/khmarbaise/maven-test-project-generator

Kind regards
Karl Heinz Marbaise

On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:

Hi to all,

i have a question concerning the memory consumption...

If i run maven with the same JDK and the same reactor and build with the
same parameter and plugins...

will the printout at the end of the build (Final Memory) something
realiable about the consumption of the JVM during the build ?...Or is it
at least a hint...or would i need to do something different (BTW:
Someone has a hint about that?) ...


[INFO] BUILD SUCCESS
[INFO]

[INFO] Total time: 6.431 s
[INFO] Finished at: 2016-04-17T17:46:58+02:00
[INFO] Final Memory: 47M/638M

So if i ran the same build with different Maven versions so could this
give us a hint where more memory is consumed ...(to identify where and
why is a different story)...

Kind regards


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Maven Memory Consumption

[Kristian Rosenvold]

I somehow assume the original patch that caused this behaviour was created
due to memory leaks in embedded mode. Do de have any way to handle both ?

(I'm wondering if we have any scope that is bound to the current embedsed
execution)

(I'm all i favour of this change no matter what...)

Kristian
1. jun. 2016 12.15 a.m. skrev "Karl Heinz Marbaise" :

> Hi,
>
> tested without the patch (-Xmx6g) ...run time for the test project more
> than two 2 Minutes
>
> running with the patch (-Xmx1g):
>
> Run time ca. 27 seconds...
>
> also worked with -Xmx768m ...ca. 30 seconds...
>
> so looks very good...
>
> Let us wait what the IT's say...
>
> Kind regards
> Karl Heinz Marbaise
>
> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
>
>> Hi,
>>
>> after more investigation and an extremly good tip of Andriy...(see
>> MNG-6030) and in the end the solution:
>>
>> Using test project with 5000 modules just doing:
>>
>> mvn clean
>>
>> using the patch now in master (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
>> extremely reduces the memory footprint...
>>
>>
>> https://github.com/khmarbaise/maven-test-project-generator/blob/master/Maven340-with-patch-5000.png
>>
>>
>> This shows the result using the patch
>>
>> The following shows Maven 3.3.9:
>>
>>
>> https://github.com/khmarbaise/maven-test-project-generator/blob/master/Maven339-5000.png
>>
>>
>> Many thanks to Andriy for the support and help...
>>
>> we will see if not IT's will fail on the change.
>>
>>
>> Kind regards
>> Karl Heinz Marbaise
>>
>> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
>>
>>> Hi,
>>>
>>> i started a little bit more detailed analysis..
>>>
>>> very simple via JConsole and running the different versions...
>>>
>>> I have summarized this here:
>>>
>>> https://github.com/khmarbaise/maven-test-project-generator
>>>
>>> Kind regards
>>> Karl Heinz Marbaise
>>>
>>> On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
>>>
 Hi to all,

 i have a question concerning the memory consumption...

 If i run maven with the same JDK and the same reactor and build with the
 same parameter and plugins...

 will the printout at the end of the build (Final Memory) something
 realiable about the consumption of the JVM during the build ?...Or is it
 at least a hint...or would i need to do something different (BTW:
 Someone has a hint about that?) ...


 [INFO] BUILD SUCCESS
 [INFO]
 
 [INFO] Total time: 6.431 s
 [INFO] Finished at: 2016-04-17T17:46:58+02:00
 [INFO] Final Memory: 47M/638M

 So if i ran the same build with different Maven versions so could this
 give us a hint where more memory is consumed ...(to identify where and
 why is a different story)...

 Kind regards

>>>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
>

Re: Maven Memory Consumption

[Manfred Moser]

I can feel your excitement coming through in the emails.. ;-) 

Karl Heinz Marbaise wrote on 2016-05-31 15:14:

> Hi,
> 
> tested without the patch (-Xmx6g) ...run time for the test project more 
> than two 2 Minutes
> 
> running with the patch (-Xmx1g):
> 
> Run time ca. 27 seconds...
> 
> also worked with -Xmx768m ...ca. 30 seconds...
> 
> so looks very good...
> 
> Let us wait what the IT's say...
> 
> Kind regards
> Karl Heinz Marbaise
> 
> On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:
>> Hi,
>>
>> after more investigation and an extremly good tip of Andriy...(see
>> MNG-6030) and in the end the solution:
>>
>> Using test project with 5000 modules just doing:
>>
>> mvn clean
>>
>> using the patch now in master (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
>> extremely reduces the memory footprint...
>>
>> https://github.com/khmarbaise/maven-test-project-generator/blob/master/Maven340-with-patch-5000.png
>>
>>
>> This shows the result using the patch
>>
>> The following shows Maven 3.3.9:
>>
>> https://github.com/khmarbaise/maven-test-project-generator/blob/master/Maven339-5000.png
>>
>>
>> Many thanks to Andriy for the support and help...
>>
>> we will see if not IT's will fail on the change.
>>
>>
>> Kind regards
>> Karl Heinz Marbaise
>>
>> On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:
>>> Hi,
>>>
>>> i started a little bit more detailed analysis..
>>>
>>> very simple via JConsole and running the different versions...
>>>
>>> I have summarized this here:
>>>
>>> https://github.com/khmarbaise/maven-test-project-generator
>>>
>>> Kind regards
>>> Karl Heinz Marbaise
>>>
>>> On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:
 Hi to all,

 i have a question concerning the memory consumption...

 If i run maven with the same JDK and the same reactor and build with the
 same parameter and plugins...

 will the printout at the end of the build (Final Memory) something
 realiable about the consumption of the JVM during the build ?...Or is it
 at least a hint...or would i need to do something different (BTW:
 Someone has a hint about that?) ...


 [INFO] BUILD SUCCESS
 [INFO]
 
 [INFO] Total time: 6.431 s
 [INFO] Finished at: 2016-04-17T17:46:58+02:00
 [INFO] Final Memory: 47M/638M

 So if i ran the same build with different Maven versions so could this
 give us a hint where more memory is consumed ...(to identify where and
 why is a different story)...

 Kind regards
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Maven Memory Consumption

[Karl Heinz Marbaise]

Hi,

tested without the patch (-Xmx6g) ...run time for the test project more 
than two 2 Minutes


running with the patch (-Xmx1g):

Run time ca. 27 seconds...

also worked with -Xmx768m ...ca. 30 seconds...

so looks very good...

Let us wait what the IT's say...

Kind regards
Karl Heinz Marbaise

On 5/31/16 10:49 PM, Karl Heinz Marbaise wrote:

Hi,

after more investigation and an extremly good tip of Andriy...(see
MNG-6030) and in the end the solution:

Using test project with 5000 modules just doing:

mvn clean

using the patch now in master (41144e7ecf52e7ec3850f3e78d81f42f505f4af8)
extremely reduces the memory footprint...

https://github.com/khmarbaise/maven-test-project-generator/blob/master/Maven340-with-patch-5000.png


This shows the result using the patch

The following shows Maven 3.3.9:

https://github.com/khmarbaise/maven-test-project-generator/blob/master/Maven339-5000.png


Many thanks to Andriy for the support and help...

we will see if not IT's will fail on the change.


Kind regards
Karl Heinz Marbaise

On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:

Hi,

i started a little bit more detailed analysis..

very simple via JConsole and running the different versions...

I have summarized this here:

https://github.com/khmarbaise/maven-test-project-generator

Kind regards
Karl Heinz Marbaise

On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:

Hi to all,

i have a question concerning the memory consumption...

If i run maven with the same JDK and the same reactor and build with the
same parameter and plugins...

will the printout at the end of the build (Final Memory) something
realiable about the consumption of the JVM during the build ?...Or is it
at least a hint...or would i need to do something different (BTW:
Someone has a hint about that?) ...


[INFO] BUILD SUCCESS
[INFO]

[INFO] Total time: 6.431 s
[INFO] Finished at: 2016-04-17T17:46:58+02:00
[INFO] Final Memory: 47M/638M

So if i ran the same build with different Maven versions so could this
give us a hint where more memory is consumed ...(to identify where and
why is a different story)...

Kind regards


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Maven Memory Consumption

[Karl Heinz Marbaise]

Hi,

after more investigation and an extremly good tip of Andriy...(see 
MNG-6030) and in the end the solution:


Using test project with 5000 modules just doing:

mvn clean

using the patch now in master (41144e7ecf52e7ec3850f3e78d81f42f505f4af8) 
extremely reduces the memory footprint...


https://github.com/khmarbaise/maven-test-project-generator/blob/master/Maven340-with-patch-5000.png

This shows the result using the patch

The following shows Maven 3.3.9:

https://github.com/khmarbaise/maven-test-project-generator/blob/master/Maven339-5000.png

Many thanks to Andriy for the support and help...

we will see if not IT's will fail on the change.


Kind regards
Karl Heinz Marbaise

On 4/22/16 9:49 PM, Karl Heinz Marbaise wrote:

Hi,

i started a little bit more detailed analysis..

very simple via JConsole and running the different versions...

I have summarized this here:

https://github.com/khmarbaise/maven-test-project-generator

Kind regards
Karl Heinz Marbaise

On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:

Hi to all,

i have a question concerning the memory consumption...

If i run maven with the same JDK and the same reactor and build with the
same parameter and plugins...

will the printout at the end of the build (Final Memory) something
realiable about the consumption of the JVM during the build ?...Or is it
at least a hint...or would i need to do something different (BTW:
Someone has a hint about that?) ...


[INFO] BUILD SUCCESS
[INFO]

[INFO] Total time: 6.431 s
[INFO] Finished at: 2016-04-17T17:46:58+02:00
[INFO] Final Memory: 47M/638M

So if i ran the same build with different Maven versions so could this
give us a hint where more memory is consumed ...(to identify where and
why is a different story)...

Kind regards


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Maven Memory Consumption

[Karl Heinz Marbaise]

Hi,

i started a little bit more detailed analysis..

very simple via JConsole and running the different versions...

I have summarized this here:

https://github.com/khmarbaise/maven-test-project-generator

Kind regards
Karl Heinz Marbaise

On 4/17/16 5:50 PM, Karl Heinz Marbaise wrote:

Hi to all,

i have a question concerning the memory consumption...

If i run maven with the same JDK and the same reactor and build with the
same parameter and plugins...

will the printout at the end of the build (Final Memory) something
realiable about the consumption of the JVM during the build ?...Or is it
at least a hint...or would i need to do something different (BTW:
Someone has a hint about that?) ...


[INFO] BUILD SUCCESS
[INFO]

[INFO] Total time: 6.431 s
[INFO] Finished at: 2016-04-17T17:46:58+02:00
[INFO] Final Memory: 47M/638M

So if i ran the same build with different Maven versions so could this
give us a hint where more memory is consumed ...(to identify where and
why is a different story)...

Kind regards
Karl Heinz

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Maven Memory Consumption

[Christian Schulte]

Am 04/18/16 um 01:39 schrieb Christian Schulte:
> Am 04/17/16 um 18:08 schrieb Bernd Eckenfels:
>> Hello,
>>
>> I wondered about that as well. It was discussed 2012 on maven-dev. The
>> statement looks like this:
>>
>> # Runtime r = Runtime.getRuntime();
>> # long MB = 1024 * 1024;
>> # "Final Memory: " + ( r.totalMemory() - r.freeMemory() ) / MB + "M/" +
>> r.totalMemory() / MB + "M"
> 
> It should be possible to get at more detailed information using the
> memory related classes of package 'java.lang.management'.
> 
> Regards,
> 

Those numbers tell you nothing about how much memory "the build" used. A
lot of things are not executed inside the Maven VM. javadoc, jarsigner,
gpg, etc.

Regards,


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Maven Memory Consumption

[Christian Schulte]

Am 04/17/16 um 18:08 schrieb Bernd Eckenfels:
> Hello,
> 
> I wondered about that as well. It was discussed 2012 on maven-dev. The
> statement looks like this:
> 
> # Runtime r = Runtime.getRuntime();
> # long MB = 1024 * 1024;
> # "Final Memory: " + ( r.totalMemory() - r.freeMemory() ) / MB + "M/" +
> r.totalMemory() / MB + "M"

It should be possible to get at more detailed information using the
memory related classes of package 'java.lang.management'.

Regards,


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Maven Memory Consumption

[Bernd Eckenfels]

Hello,

I wondered about that as well. It was discussed 2012 on maven-dev. The
statement looks like this:

# Runtime r = Runtime.getRuntime();
# long MB = 1024 * 1024;
# "Final Memory: " + ( r.totalMemory() - r.freeMemory() ) / MB + "M/" +
r.totalMemory() / MB + "M"

So this basically prints the currently used memory (which is pretty
useless as it depends on the last time the GC was run) and the memory
commit. This is the largest memory needed before a GC was kicked off.
The later one should be a pretty usefull indicator of the commited
memory. It is as least as big as Xms, so you should set it low.

It is however not an indication if you can decrease Xmx. If it tells
you the total memory commited was 1,5GB it is possible that it would
run with Xmx 1G as well.

Gruss
Bernd


 Am Sun, 17 Apr 2016 17:50:05 +0200
schrieb Karl Heinz Marbaise :

> Hi to all,
> 
> i have a question concerning the memory consumption...
> 
> If i run maven with the same JDK and the same reactor and build with
> the same parameter and plugins...
> 
> will the printout at the end of the build (Final Memory) something 
> realiable about the consumption of the JVM during the build ?...Or is
> it at least a hint...or would i need to do something different (BTW: 
> Someone has a hint about that?) ...
> 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 6.431 s
> [INFO] Finished at: 2016-04-17T17:46:58+02:00
> [INFO] Final Memory: 47M/638M
> 
> So if i ran the same build with different Maven versions so could
> this give us a hint where more memory is consumed ...(to identify
> where and why is a different story)...
> 
> Kind regards
> Karl Heinz
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Maven Memory Consumption

[Karl Heinz Marbaise]

Hi to all,

i have a question concerning the memory consumption...

If i run maven with the same JDK and the same reactor and build with the 
same parameter and plugins...


will the printout at the end of the build (Final Memory) something 
realiable about the consumption of the JVM during the build ?...Or is it 
at least a hint...or would i need to do something different (BTW: 
Someone has a hint about that?) ...



[INFO] BUILD SUCCESS
[INFO] 


[INFO] Total time: 6.431 s
[INFO] Finished at: 2016-04-17T17:46:58+02:00
[INFO] Final Memory: 47M/638M

So if i ran the same build with different Maven versions so could this 
give us a hint where more memory is consumed ...(to identify where and 
why is a different story)...


Kind regards
Karl Heinz

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org