Hi all,
This has bugged me for years, and recent traffic brings it back to the
forefront. I apologize that this email is more a teaser than a complete
solution, but that's what time allows for right now.
In my opinion, a lot of the complexity in ASDF is caused by its conflation
of two separate goals: a "build system" and a "loader".
There are a wide variety of build tools available for C/C++ and many other
languages: make, imake, cmake, (s)cons, (b)jam, ninja, redo, etc. While
the added overhead of installing a build system before compiling a project
has drawbacks, many of these tools have compatibility with make, and most
users don't compile most code. The compatibility of files produced by
these diverse tools allows a lot of experiments to be run, and successful
features often get absorbed into tools like GNU make.
This compatibility is allowed because the loader (dynamic linker in C
speak, class loader in Java) has a simple, well-defined protocol for
telling it where to find things, independent of the build system. Look at
the linux manpages for ld.so and ldconfig...
If CL is to grow from a dev-centric ecosystem to have a large base of
"normal users", I think we need to start finding ways to enable better
long-term stability. Pre-tested, "binary" releases are exactly what an
end user wants. Applications need to "just work" and not break whenever a
dependency changes its API.
What does this mean for ASDF?
Since many CL developers want features like load-time recompilation, the
simple static decoupling in C between link and load is not possible. A
richer interface is required.
While deprecated, the CL PROVIDE and REQUIRE API is very close to ideal
for a loader. Obvious extensions include protocols for adding versioning
checks/negotiation, search paths, re-build hooks, something better than
*MODULES*, and implementation independent behavior. This API should be as
small as possible, truly abstracting the different activities, and
allowing multiple implementations to coexist.
Then tools like ASDF could produce "installs" that fit into this
framework. The core loader would still be integrated into each CL distro,
but the build tool, with its dependency tracking, source file searching,
and other features could be distributed separately, and multiple tools
could coexist in a single image.
Wherever possible, the simplest loader solution should be chosen, leaving
sysadmins and/or build tools to handle any "difficult" configuration (such
as hidden directories). For example, the loader could require that all
libraries be installed using some UUID or hash-based scheme in a common
location, and hooks inside these installs are used to activate fancy
builder-specific features.
I apologize, what I just wrote looks way too complicated. For example, it
is possible that versioning, searching, rebuilding, and loading could be a
builder-specific behavior inside a single try-load function. However it
works out, I do believe that a simple decoupled API that breaks ASDF into
a simple, stable loader core and a more featureful, faster changing piece,
would be good for everyone in the long term.
Loaders have a simpler required feature set than build systems. Loaders
just need to allow for a few types of search scoping (per system, per
user, and per program). Build systems require configuration, searching,
incremental rebuilds, installation, unit testing, etc.
Later,
Daniel
