This feature caches Tests results too, but granularity is per-project. In our 
case we even cache result of long-running integration tests as well so 
developers rerun unit/integration tests only for changed projects and 
dependents from them. That’s significant time win.
In our multi module project if you change just 1 module of 600, you invalidate 
only that module and run test for that module.

If project is invalidated by changed inputs – it will be rebuild from scratch 
by usual plugins regardless of which features are supported by particular 
plugin. In case of Takari and incremental compiler that will result in faster 
compilation for second and consequent runs.
Any plugin could benefit from cache/incremental nature for multi-module project.

Kindly yours
Aleks

From: Tibor Digana [mailto:tibordig...@apache.org]
Sent: Friday, September 13, 2019 5:18 PM
To: Maven Developers List <dev@maven.apache.org>
Cc: Alexander Ashitkin <alexander.ashit...@db.com>
Subject: Re: [VOTE] Maven incremental build for BIG-sized projects with local 
and remote caching

In theory, the incremental compiler would make it faster.
But this can be told only if you present a demo project with has trivial tests 
taking much less time to complete than the compiler.

In reality the tests in huge projects take significantly longer time than the 
compiler.
Some developers say "switch off all the tests" in the release phase but that's 
wrong because then the quality goes down and methodologies are broken.

I can see a big problem that we do not have an interface between Surefire and 
Compiler plugin negotiating which tests have been modified including modules 
and classes in the entire structure.

Having incremental compiler is easy, just use compiler:3.8.1 or use the Takari 
compiler.
But IMO the biggest benefit in performance would be after having the truly 
incremental test executor.

On Fri, Sep 13, 2019 at 10:46 PM Maximilian Novikov 
<maximilian.novi...@db.com<mailto:maximilian.novi...@db.com>> wrote:
Hi All,

We want to create upstream change to Maven to support true incremental build 
for big-sized projects.
To raise a pull request we have to pass long chain of Deutsche Bank’s internal 
procedures. So, before starting the process we would like to get your feedback 
regarding this feature.

Motivation:

Our project is hosted in mono-repo and contains ~600 modules. All modules has 
the same SNAPSHOT version.
There are lot of test automation around this, everything is tested before merge 
into release branch.

Current setup helps us to simplify build/release/dependency management for 10+ 
teams those contribute into codebase. We can release everything in 1-click.
The major drawback of such approach is build time: full local build took 45-60 
min (-T8), CI build ~25min(-T16).

To speed-up our build we needed 2 features: incremental build and shared cache.
Initially we started to think about migration to Gradle or Bazel. As migration 
costs for the mentioned tools were too high, we decided to add similar 
functionality into Maven.

Current results we get: 1-2 mins for local build(-T8) if build was cached by 
CI, CI build ~5 mins (-T16).

Feature description:

The idea is to calculate checksum for inputs and save outputs in cache.
Each node checksum calculated with:


•         Effective POM hash

•         Sources hash

•         Dependencies hash (dependencies within multi-module project)

Project sources inputs are searched inside project + all paths from plugins 
configuration:
How does it work in practice:



1.       CI: runs builds and stores outputs in shared cache

2.       CI: reuse outputs for same inputs, so time is decreasing

3.       Locally: when I checkout branch and run ‘install’ for whole project, I 
get all actual snapshots from remote cache for this branch

4.       Locally: if I change multiple modules in tree, only changed subtree is 
rebuilt

Impact on current Maven codebase is very localized (MojoExecutor, where we 
injected cache controller).
Caching can be activated/deactivated by property, so current maven flow will 
work as is.

And the big plus is that you don’t need to re-work your current project. 
Caching should work out of box, just need to add config in .mvn folder.

Please let us know what do you think. We are ready to invest in this feature 
and address any further feedback.

Kind regards,
Max



---
This e-mail may contain confidential and/or privileged information. If you are 
not the intended recipient (or have received this e-mail in error) please 
notify the sender immediately and delete this e-mail. Any unauthorized copying, 
disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and 
regulatory disclosures and to 
http://www.db.com/unitedkingdom/content/privacy.htm for information about 
privacy.


---
Die Europäische Kommission hat unter http://ec.europa.eu/consumers/odr/ eine 
Europäische Online-Streitbeilegungsplattform (OS-Plattform) errichtet. 
Verbraucher können die OS-Plattform für die außergerichtliche Beilegung von 
Streitigkeiten aus Online-Verträgen mit in der EU niedergelassenen Unternehmen 
nutzen.

Informationen (einschließlich Pflichtangaben) zu einzelnen, innerhalb der EU 
tätigen Gesellschaften und Zweigniederlassungen des Konzerns Deutsche Bank 
finden Sie unter https://www.deutsche-bank.de/Pflichtangaben. Diese E-Mail 
enthält vertrauliche und/ oder rechtlich geschützte Informationen. Wenn Sie 
nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, 
informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Das 
unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht 
gestattet.

The European Commission has established a European online dispute resolution 
platform (OS platform) under http://ec.europa.eu/consumers/odr/. Consumers may 
use the OS platform to resolve disputes arising from online contracts with 
providers established in the EU.

Please refer to https://www.db.com/disclosures for information (including 
mandatory corporate particulars) on selected Deutsche Bank branches and group 
companies registered or incorporated in the European Union. This e-mail may 
contain confidential and/or privileged information. If you are not the intended 
recipient (or have received this e-mail in error) please notify the sender 
immediately and delete this e-mail. Any unauthorized copying, disclosure or 
distribution of the material in this e-mail is strictly forbidden.

Reply via email to