On Sun, Apr 15, 2012 at 10:57 PM, Charles Smith <[email protected]> wrote:
> To what ends, is the ability to do separate compilation NECESSARY?
> Please also construct and deconstruct the possible counter-arguments.
>
Bill Leslie raises some interesting issues, but the short answer is that we
need the ability to do dynamic libraries. I am prepared to accept the
proposition that "assemblies" should be whole-program compiled *up to* the
assembly boundary. An "assembly" is basically a user-defined unit of code
that is loaded dynamically and has a well-defined external perimeter at
which it is linked to other assemblies at run time in order to form a whole
program. So a dynamic library is an assembly. The program that loads the
dynamic library is also an assembly.
The problem that Bill raises on his 600KSLOC application is very real.
Admittedly it sounds like his compiler is genetically related to a
tortoise, but in the final analysis *any* system that relies on
compile-time code expansion *over unboxed types* has to pay the piper in
some fashion. From a "lines of code" perspective, a template declaration
[C++] or a generic class [Java] or a parametric declaration [BitC] is
effectively implemented by macro-expansion when unboxed types get involved.
The intrusion is much less for boxed types (i.e. types whose instances live
in the heap).
The heart of the problem lies in two places:
1. Combinatorics at ground types.
2. Parametric declarations that are exposed at assembly boundaries.
The combinatorics problem becomes clear when we note that BitC has 11
distinct ground operators for addition. If we intend to specialize these
aggressively, then when the compiler sees:
def illustrative_add(a, b) { a + b }
Arith('a) => 'a x 'a -> 'a
we need to generate 11 distinct implementations. For less contrived
examples that can lead to a lot of code multiplication in a shared library
assembly (though it doesn't tend to affect the main application assembly).
We don't have hard numbers on the impact, but I suspect that it's high.
And a lot of it turns out to be pointless. The type:
union List('a) is
cons: { car: 'a; cdr: List 'a }
nil
hypothetically needs to be expanded for every type 'a, but can actually use
the same type for any type 'a that is a boxed type.
I actually suspect that we could survive this issue, mainly because dynamic
libraries don't tend to use a lot of underspecified ground operations on
ground types, and those that do are willing to expose the code in
header-equivalent files.
Which brings me to parametric declarations. The long and short of it comes
down to a simple pair of statements:
1. The *purpose* of a shared library, in the main, is to obtain code
sharing
2. Late-bound parametric procedures cannot easily be generated before
dynamic link time, and therefore cannot be shared.
So, for example, if we put our BTree('a) implementation into the standard
library, we aren't going to get to share the code unless we do
hand-directed instantiations at popular types. Which isn't necessarily a
bad thing.
The kicker is that we rely on dynamic library boundaries for other things
as well: most notably for bug fixes. From this perspective, we really *don't
* want to compile too eagerly, because we want to update the code when a
new library version is installed. The earliest point at which we can do
that compilation is when the relevant assembly update is installed.
And that may actually hint at a viable solution. We can imagine a
compilation model in which JIT is used to instantiate late-bound
procedures, but in which we notice these operations and update the DLL on
the machine in the field to add newly instantiated procedures to the DLL
when they turn out to be used by multiple applications. Think of it as a
severely delayed form of AOT, or (equivalently) as a way of selectively
caching JIT results across executions of multiple applications.
On Sun, Apr 15, 2012 at 10:57 PM, Charles Smith <[email protected]> wrote:
> Note: I am not challenging the judgement (I agree for reasons partially
> rational and partially emanating from personal history.)
>
Feel free to question the judgment. There's a lot of "wisdom from
experience" running around in this space that guides us astray until we
look deeply enough into the problem to realize that it may not apply.
shap
_______________________________________________
bitc-dev mailing list
[email protected]
http://www.coyotos.org/mailman/listinfo/bitc-dev
