Hi, Interesting. I created MNG-6571 to track this issue: please move the memory dump on this issue. Then I created MNG-6571 branch [2] with a cache on VersionRange: this class is currently immutable, then caching it is trivial. I don't know if it's the best location, or if there are not other locations where a cache would make sense, but it was an easy and safe try.
Can you build this branch for yourself and share a new memory dump, please? Regards, Hervé [1] https://issues.apache.org/jira/browse/MNG-6571 [2] https://github.com/apache/maven/tree/MNG-6571 Le mercredi 23 janvier 2019, 10:10:40 CET Dameron, Yann a écrit : > Hi, > > > > I'm working on a huge project with plenty modules and we are facing a memory > consumption issue. We observed it when Maven computes the reactor and then > during the build itself. > > > > Quick summary of our situation (as we can't share the POMs) > > * We have a dedicated POM to manage all versions (3rd parties and our > own modules). This POM is split in 5 sub poms to organize them a bit > (apache, netbeans, xxx, ...) > > On this POM structure, we declare 3049 <dependency>...</dependency>. > > * We have a parent pom. This one is used by all our 1628 modules. It > defines the plugins versions and import the dependency management > (explained above). * We have 1628 modules which are using our parent pom > (and then inherit the dependency management imported by the parent) * We > have an aggregator POMs structure which is responsible to include all the > modules. Root aggregator is the one on which the full rebuild is triggered. > > > > When Maven computes the reactor, it consumes a huge amount of memory (8 - 12 > Gb) > > > > I made a memory dump just before the end of reactor computation. I attached > a screenshot in MNG-5661<https://issues.apache.org/jira/browse/MNG-5661> as > I feel it's related, please remove it if it's not appropriated. On this > screenshot we clearly see from where come from our memory issues: > https://issues.apache.org/jira/secure/attachment/12955925/Maven-Reactor-Dum > p.png > > > > The number of Dependency instance seems to be related to our structure. All > our projects import our big dependency management (inherited from the > parent) so we have 1628 x 3049 => ~ 5 000 000 instances. > > This is also approximately the number of MavenProject, DefaultArtifact, > VersionRange, etc... so I feel that the memory consumption comes from the > fact that Maven doesn't detect such cases and duplicate everything. > > > > Is Maven optimized to detect such cases? It would be great to ensure that > Dependency (and others) instances are not duplicated. > > > > Thanks, > Yann > > Information in this e-mail and any attachments is confidential, and may not > be copied or used by anyone other than the addressee, nor disclosed to any > third party without our permission. There is no intention to create any > legally binding contract or other binding commitment through the use of > this electronic communication unless it is issued in accordance with the > Experian Limited standard terms and conditions of purchase or other express > written agreement between Experian Limited and the recipient. Although > Experian has taken reasonable steps to ensure that this communication and > any attachments are free from computer viruses, you are advised to take > your own steps to ensure that they are actually virus free. > > Experian Ltd is authorised and regulated by the Financial Conduct Authority. > Companies Act information: Registered name: Experian Limited. Registered > office: The Sir John Peace Building, Experian Way, NG2 Business Park, > Nottingham, NG80 1ZZ, United Kingdom. Place of registration: England and > Wales. Registered number: 653331. > > Vos informations nominatives sont exploitées par la société Scorex S.A.M. > (Groupe Experian) dans le cadre du traitement ayant pour finalité "Gestion > de la messagerie électronique professionnelle". Conformément à la loi n° > 1.165, modifiée, vous disposez d'un droit d'accès, de rectification et de > suppression en écrivant au Secrétariat Général de la société Scorex, sise, > 2, rue de la Lüjerneta à Monaco (98000). > > Your personal information is used by Scorex S.A.M. (Experian Group) as part > of the processing for the purpose of "Professional e-mail management". In > accordance with the law n ° 1.165, modified, you have a right of access, > correction and suppression by writing to the General Secretariat of the > company, located, 2, rue de la Lüjerneta in Monaco (98000). --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
