Hi Alexander and Maximilian. As a former Deutsche Bank RTC employee, I cannot bypass this topic.
First of all, I'm not speaking for community, neither Jetbrains. I know how painfull is to get all approvals for open-source contribution in Deutsche Bank, but may be you can share you solution as a maven extension first? May be I've missed link to github, but discussion looks strange for me - we are discussing pros and cons of solution, which is not public yet. >so any ide with maven integration should benefit from cache them maven action invoked AFAIK, m2e use its own lifecycle (please, correct me, if I'm wrong). Not sure, how this change will affect it's performance. But IDEA executes maven tasks (actually, in a quite hacky-way) for resolving dependencies, source folders, artifacts, etc. Compiler plugin is not being runned usually by IDEA Compilation and tests runs of imported maven projects are performed by IDEA itself, and IDEA does not use maven compilation results when run tests. No maven code and maven extensions are used after importing, so for IDEA this change should be irrelevant, as I got it (the only exception case is if you use "delegate maven" option, but this is a point of another discussion) Thanks, Alex. On Thu, Sep 19, 2019 at 8:48 AM Alexander Ashitkin <ashitkin.a...@gmail.com> wrote: > Sorry if duplicated, looks like my yesterday reply didn't come through. > Sharing results. > > Configuration: > * verify -T4 -P default,all-shapshots-repos > * my project config (might be suboptimal for wicket) > * scala tests disabled in 2 modules (caused bytecode version conflict on > my machine) > > Results > Clean state (cache disabled): 15:58 min > Second run, target up to date (cache disabled): 10:20 min > Fully cached (no changes): 17.507 s > wicketstuff-jwicket-tooltip-wtooltips changed: 34.936 s > wicketstuff-rest-utils changed: 54.040 s > > > For wicketstuff-jwicket-tooltip-wtooltips i didnt check invalidated > modules, for wicketstuff-rest-utils > [wicketstuff-rest-lambda, wicketstuff-restannotations, > wicketstuff-restannotations-json, wicketstuff-restannotations-examples] > were invalidated and rebuilt > > If you want to try other modules - please let me know. > > regarding ide - it's a usual maven installation, so any ide with maven > integration should benefit from cache them maven action invoked > > Thank you > Aleks > > > On 2019/09/17 12:29:11, Martijn Dashorst <martijn.dasho...@gmail.com> > wrote: > > This seems like it would benefit a lot of projects (at least it would > ours). > > > > How would this work in coordination with IDE's? m2e has (afaict, but > > haven't looked closely) its own lifecycle management to bridge eclipse > and > > maven. AFIAK only Netbeans uses maven directly? > > > > If you want to benchmark a public big repo, you can use Wicket Stuff > Core ( > > https://github.com/wicketstuff/core). It has 237 modules, and the build > > takes quite a while to compile and package. The project levels are not > > deep, but there's some nesting. > > > > Martijn > > > > > > On Fri, Sep 13, 2019 at 10:46 PM Maximilian Novikov < > > maximilian.novi...@db.com> wrote: > > > > > Hi All, > > > > > > > > > > > > *We want to create upstream change to Maven* to support true > incremental > > > build for big-sized projects. > > > > > > To raise a pull request we have to pass long chain of Deutsche Bank’s > > > internal procedures. So, *before starting the process we would like to > > > get your feedback regarding this feature*. > > > > > > > > > > > > *Motivation:* > > > > > > > > > > > > Our project is hosted in mono-repo and contains ~600 modules. All > modules > > > has the same SNAPSHOT version. > > > > > > There are lot of test automation around this, everything is tested > before > > > merge into release branch. > > > > > > > > > > > > Current setup helps us to simplify build/release/dependency management > for > > > 10+ teams those contribute into codebase. We can release everything in > > > 1-click. > > > > > > The major drawback of such approach is build time: *full local build > took > > > 45-60 min (*-T8)*, CI build ~25min(*-T16*)*. > > > > > > > > > > > > To speed-up our build we needed 2 features: incremental build and > shared > > > cache. > > > > > > Initially we started to think about migration to Gradle or Bazel. As > > > migration costs for the mentioned tools were too high, we decided to > add > > > similar functionality into Maven. > > > > > > > > > > > > Current results we get: *1-2 mins for local build(*-T8*)* if build was > > > cached by CI*, CI build ~5 mins (*-T16*).* > > > > > > > > > > > > *Feature description:* > > > > > > > > > > > > The idea is to calculate checksum for inputs and save outputs in cache. > > > > > > [image: image2019-8-27_20-0-14.png] > > > > > > Each node checksum calculated with: > > > > > > > > > > > > · Effective POM hash > > > > > > · Sources hash > > > > > > · Dependencies hash (dependencies within multi-module project) > > > > > > > > > > > > Project sources inputs are searched inside project + all paths from > > > plugins configuration: > > > > > > [image: image2019-8-30_10-28-56.png] > > > > > > How does it work in practice: > > > > > > > > > > > > 1. CI: runs builds and stores outputs in shared cache > > > > > > 2. CI: reuse outputs for same inputs, so time is decreasing > > > > > > 3. Locally: when I checkout branch and run ‘install’ for whole > > > project, I get all actual snapshots from remote cache for this branch > > > > > > 4. Locally: if I change multiple modules in tree, only changed > > > subtree is rebuilt > > > > > > > > > > > > Impact on current Maven codebase is very localized (MojoExecutor, > where we > > > injected cache controller). > > > > > > Caching can be activated/deactivated by property, so current maven flow > > > will work as is. > > > > > > > > > > > > And the big plus is that you don’t need to re-work your current > project. > > > Caching should work out of box, just need to add config in .mvn folder. > > > > > > > > > > > > Please let us know what do you think. We are ready to invest in this > > > feature and address any further feedback. > > > > > > > > > > > > Kind regards, > > > > > > Max > > > > > > > > > > > > > > > --- > > > This e-mail may contain confidential and/or privileged information. If > you > > > are not the intended recipient (or have received this e-mail in error) > > > please notify the sender immediately and delete this e-mail. Any > > > unauthorized copying, disclosure or distribution of the material in > this > > > e-mail is strictly forbidden. > > > > > > Please refer to https://www.db.com/disclosures for additional EU > > > corporate and regulatory disclosures and to > > > http://www.db.com/unitedkingdom/content/privacy.htm for information > about > > > privacy. > > > > > > > > > -- > > Become a Wicket expert, learn from the best: http://wicketinaction.com > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > -- Alexander Bubenchikov Software developer JetBrains http://www.jetbrains.com The Drive to Develop
