Hi Yes we tried, but Takari is a bit different story – it’s a smarter scheduler which gives you some boost over default lifecycle scheduler, but still require you to build your modules. This feature is true incremental build – you don’t build modules which were not changed at all and build only modified/changed ones. Required build state for skipped modules is restored from cache. So for our 600 modules build time is down to 1 minute from ~40 minutes and even single threaded build benefits from the cache. Takari just doesn’t do that.
Kindly yours Aleks From: Tamás Cservenák [mailto:ta...@cservenak.net] Sent: Friday, September 13, 2019 4:54 PM To: Maven Developers List <dev@maven.apache.org> Cc: Alexander Ashitkin <alexander.ashit...@db.com> Subject: Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching Hi there, just a shot in a dark: Have you tried any of the existing stuff, like Takari Lifecycle before modding Maven itself? (http://takari.io/book/40-lifecycle.html) Thanks, T On Fri, Sep 13, 2019 at 10:46 PM Maximilian Novikov <maximilian.novi...@db.com<mailto:maximilian.novi...@db.com>> wrote: Hi All, We want to create upstream change to Maven to support true incremental build for big-sized projects. To raise a pull request we have to pass long chain of Deutsche Bank’s internal procedures. So, before starting the process we would like to get your feedback regarding this feature. Motivation: Our project is hosted in mono-repo and contains ~600 modules. All modules has the same SNAPSHOT version. There are lot of test automation around this, everything is tested before merge into release branch. Current setup helps us to simplify build/release/dependency management for 10+ teams those contribute into codebase. We can release everything in 1-click. The major drawback of such approach is build time: full local build took 45-60 min (-T8), CI build ~25min(-T16). To speed-up our build we needed 2 features: incremental build and shared cache. Initially we started to think about migration to Gradle or Bazel. As migration costs for the mentioned tools were too high, we decided to add similar functionality into Maven. Current results we get: 1-2 mins for local build(-T8) if build was cached by CI, CI build ~5 mins (-T16). Feature description: The idea is to calculate checksum for inputs and save outputs in cache. Each node checksum calculated with: • Effective POM hash • Sources hash • Dependencies hash (dependencies within multi-module project) Project sources inputs are searched inside project + all paths from plugins configuration: How does it work in practice: 1. CI: runs builds and stores outputs in shared cache 2. CI: reuse outputs for same inputs, so time is decreasing 3. Locally: when I checkout branch and run ‘install’ for whole project, I get all actual snapshots from remote cache for this branch 4. Locally: if I change multiple modules in tree, only changed subtree is rebuilt Impact on current Maven codebase is very localized (MojoExecutor, where we injected cache controller). Caching can be activated/deactivated by property, so current maven flow will work as is. And the big plus is that you don’t need to re-work your current project. Caching should work out of box, just need to add config in .mvn folder. Please let us know what do you think. We are ready to invest in this feature and address any further feedback. Kind regards, Max --- This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy. --- Die Europäische Kommission hat unter http://ec.europa.eu/consumers/odr/ eine Europäische Online-Streitbeilegungsplattform (OS-Plattform) errichtet. Verbraucher können die OS-Plattform für die außergerichtliche Beilegung von Streitigkeiten aus Online-Verträgen mit in der EU niedergelassenen Unternehmen nutzen. Informationen (einschließlich Pflichtangaben) zu einzelnen, innerhalb der EU tätigen Gesellschaften und Zweigniederlassungen des Konzerns Deutsche Bank finden Sie unter https://www.deutsche-bank.de/Pflichtangaben. Diese E-Mail enthält vertrauliche und/ oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet. The European Commission has established a European online dispute resolution platform (OS platform) under http://ec.europa.eu/consumers/odr/. Consumers may use the OS platform to resolve disputes arising from online contracts with providers established in the EU. Please refer to https://www.db.com/disclosures for information (including mandatory corporate particulars) on selected Deutsche Bank branches and group companies registered or incorporated in the European Union. This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.
