I also support exploring a move away from Apache Maven for orchestrating our build.
For a single-module project, I still think it can be a good build tool, and we could still use it for this sort of thing, but I think we are reaching a multi-module scale where it does not work well. Almost all of our jobs build things that are not needed and run tests that are redundant, and it is not easy to do better, even with a sequence of maven commands. I'd like to lay out what we hope for from a tool. Here's a start: General: - Dependency-driven build so devs working on one thing build & test only what is needed - Supports orchestration across Protobuf, Java, Python, Go, Docker images - Meets devs where they are, letting folks in one language use familiar tools - Caching across builds as much as possible - Easily extensible for when it doesn't have the feature we need (writing a maven plugin is too much, using maven-exec-plugin is too crufty) - Preferably a declarative configuration language Java needs beyond the basics, which could be executed by the orchestrator or my module-local mvn builds, etc. - Pulling deps from maven central and alternate repos - Findbugs - RAT - Dependency rule enforcement - IWYU (Include What You Use) - Multiple Java versions in same project - ASF release workflow I probably missed some must-haves or nice-to-haves. I'd love to compile thoughts on other languages' needs. Based on these, another project I would consider is Bazel. We could very easily use it to orchestrate, but use Maven (or Gradle!) on the leaves. I also think that Gradle is also more focused on the JVM ecosystem, so it is not quite as neutral as Bazel, and uses Groovy which is a bit more esoteric than Python for writing Bazel rules. Kenn On Mon, Oct 30, 2017 at 9:37 AM, Lukasz Cwik <lc...@google.com.invalid> wrote: > I wanted to make this thread more visible. This discussion stems from Ken's > thread about Jenkins pre/post commit issues[1]. > > I did some investigation as for ways to improve the quality of the signal > from Jenkins by trying to modify the Jenkins jobs spawned from Groovy. I > had limited success but everything I felt like I was doing was just > patching symptoms of the problem which is that our build is just too slow. > For example, we keep adding all these profiles to Maven or tuning how a > plugin runs to eek out a small decrease in build time. I believe swapping > away from Apache Maven to a build tool which only builds the things which > have changed in a PR would be the best approach. > > I would suggest that we migrate to Gradle as our build tool. I am > suggesting Gradle because: > * It is used in lots of open source projects and has a very large community > behind it. > * It has better support for building languages other then Java > (PyGradle[2], GoGradle[3], ...) > * Its incremental build support works and only builds things that changed > through the use of a build cache. Even without the build cache (or for > clean builds), it is much faster. > * Apache Infra already has Gradle v4.x installed on the Jenkins machines. > > Any alternatives that should be considered or should we stick with Apache > Maven? > > 1: > https://lists.apache.org/thread.html/25311e0e95be5c49afb168d9b4b4d3 > 57984c10c39c7b01da8ff3baaf@%3Cdev.beam.apache.org%3E > 2: https://github.com/linkedin/pygradle > 3: https://github.com/gogradle/gogradle >
