On Monday, 12 February 2018 at 10:35:06 UTC, Russel Winder wrote:
In all the discussion of Dub to date, it hasn't been pointed out that JVM building merged dependency management and build a long time ago. Historically:

  Make → Ant → Maven → Gradle

and Gradle can handle C++ as well as JVM language builds.

So the integration of package management and build as seen in Go, Cargo, and Dub is not a group of outliers. Could it be then that it is the right thing to do. After all package management is a dependency management activity and build is a dependency management activity, so why separate them, just have a single ADG to describe the whole thing.

SCons, CMake, and Meson (also Reggae?) are traditional build tools, but they assume all package dependency management is handled elsewhere, i.e. they state what is required for a build but assume some other tool provides those packages, usually OS package management, but note with C++ Conan is rapidly becoming a big player. JFrog via Bintray and Artifactory do appear to be the leaders in repository management and Gradle works bery well with it.

Rust, Ceylon, and D have to date chosen to eschew systems like Bintray to create their own language specific versions for whatever reason. This leads to language specific dependency management and build. Go is also in this category really except that Git, Mercurial, and Breezy (née Bazaar) repositories are the only package storage used.

Then, is a DevOps world, there is deployment, which is usually a dependency management task. Is a totally new tool doing ADG manipulation really needed for this?

The lessons from all the tools from SCons to Gradle is that it is all about ADG manipulation and constraint satisfaction. SCons is really quite restricted even though it does it very well (*). Gradle really tries hard not just to solve the problems of Maven (**), but to do end- to-end project management well. In a sense it is the antithesis of each tool does one thing and one thing only model, it is the "there is one and only one ADG to describe the life of the project". Maven and Gradle, and to a lesser extent Cargo and Go, emphasise project management as a wholistic thing, rather than making people deside on each item of the tool chain. Gradle proves a good plugin system to allow changes to the default standard project lifecycle.

Gradle uses Groovy scripts or Kotlin scripts for project specifications. Most projects are easily described by a purely declarative specification using internal DSLs. However, for those awkwards bits for some projects a bit of programming in the specification solves the problem. So this goes against the "a project specification must be purely declarative so use TOML/JSON/SDL" but is easy DevOps worth it.

Atila's Reggae has already shown how easy it is to use D (or Python, Ruby, JavaScript, Lua) to define a build in an internal DSL.

Merging ideas from Dub, Gradle, and Reggae, into a project management tool for D (with C) projects is relatively straightforward of plan albeit really quite a complicated project. Creating the core ADG processing is the first requirement. It has to deal with external dependencies, project build dependencies, and deployment dependencies. The initial problem is creating an ADG from what is potentially a CDG and then doing constraint satisfaction to generate actions. SCons and Gradle have a lot to offer on this.

Having been obsessed by build and project management since about 1976, I'd be interested in doing some work on this.

(*) The O(N) vs. O(n), SCons vs. Tup thing that T raised in another thread is important, but actually it is an implementation thing of how do you detect change, it isn't an algorithmic issue at a system design
level. But it is important.

(**) Which many people ignore because Maven remains the major project management tool in the JVM-verse.

Lot's of stuff here. I would love for a build/package management tool to emerge and take over like git did with source control. However, my guess is that this problem is very hard and that's why there's so many tools that have their own pros and cons. But it sounds like you're trying to sift through them all and find the gems in each tool.

I myself have made attempts to tackle the problem. I created a tool called "dbuild" that allowed you to use D code to describe your project and then build it for you (kinda like Raggae I suppose). And my latest project I call "bidmake" which is more general. It implements the model of what I call "build contracts". A project describes a "contract" and various "contractors" can be used to fulfill the contract. You could have a "build contractor" that builds a contract, or a "documentation contractor" that generates documentation, etc. Anyway, I currently use the tool successfully on a few projects but I'm still not sure how much potential it has. Maybe with your experience you can take a look at it and see if there's any substance to the idea. (https://github.com/marler8997/bidmake)

Reply via email to