Hi Ian and all!
I abandoned (for now) my multithreading efforts, as the easiest way to
implement them is if there is proper support for precise object
tracing and relocation. Such a feature would trivially allow any
application to stop-and-copy all the objects in its world, modifying
them along the way to prepend any necessary metadata. Multithreading
would thus be bootstrapped out of an initially single-threaded
program.
I'm withdrawing my former "extensible metadata" proposal, because it
was overly complex, and I've since come up with some tricks to get
further towards what I'm looking for (and what seems more in line with
your thinking, from e-mails you've sent to this list). You can see
this e-mail as another of my pseudo-proposals (i.e. I'm explaining
what I want to do, but I'm not really waiting for approval before I
start digging into the code).
Zones
*****
I'm now of the belief that extensible metadata really is just having a
"memory zone" oop before the vtable oop. Then, the zone is just a
plain object that can define what per-object metadata prepends the
zone oop. So, we have:
| ...zone-specific metadata |
+---------------------------+
| ZONE-PTR & ~1 |---> zone object
+---------------------------+
| VTABLE-PTR |---> vtable object
+===========================+
| ...members (if any) |
I say "ZONE-PTR & ~1" to indicate that the low bit of the field should
be ignored when dereferencing the pointer. This way, that bit can be
used to mark objects during a traversal, without having to reserve a
separate field in the metadata (or slowing down the VTABLE-PTR
dereference, which will happen much more frequently).
What makes this even more attractive is that we can define various
zone methods on UndefinedObject, and then use nil as a zone for
objects that are statically mapped in the executable, such as
plain-old-C functions and static structs.
OOP or non-OOP
**************
It may or may not be obvious to other readers that Idst uses quite an
interesting underscore naming convention: when an oop is not really a
valid object (i.e. it interfaces with C as a pointer to raw memory, or
an untagged int), it has an underscore prepended to it. When a method
takes an argument that isn't a valid object, the selector has an
underscore appended to the appropriate argument component. When a
method returns an invalid object, it has an underscore prepended.
MyObj : Object ( anObj _notAnObj )
MyObj _returnsSomethingNotAnObject { return (oop)42; }
MyObj takesAnObject: obj andNot_: _noObj
[ ...
{ printf("%d\n", (int)v__noObj); }.
]
An object trace mechanism can exploit this convention by tracing only
members that have no leading underscore.
One of the dreams I remember you mentioning was to make functions into
proper objects (with a header), but that there was no way to do so in
plain-old-C. Because of that, every member that is a C function has
an underscore prepended, such as "closure _method". This is no good,
especially during implementation of the trace mechanism, when some
functions may be in plain-old-C (which should be treated specially,
either as "don't change" because we don't need to relocate them, or
"throw an error" because we can't serialise them), and others may be
proper objects that need tracing.
In order to provide this feature now, I simply compile all the C code
with -falign-functions=2 (or any multiple of 2), then tag them by
setting the low bit before I store them in another object. Le voila!
C functions are now proper oops (just SmallIntegers), and I can write
methods for _closure or StaticBlockClosure or whatever to handle
SmallIntegers specially. Now I'm in a position to modify Jolt to emit
functions that *are* proper objects.
_imp_t goes away, since they all are oops. The penalty: during this
interim bootstrap step, every call to such an oop method needs to "&
~1" it first (jeez, I'm sounding like a broken record). On i386,
that's another 3 bytes per call (assuming the function address was
already sitting in a register).
There's just one more issue: are anonymous temporaries (like _1, _2,
etc.) used by idc to implement chained sends oops or non-oops? We can
infer from most static selector names what is correct for either
return values or arguments. However, some methods by nature are
ambiguous and can prevent inference, such as BlockClosure #value or
_object #perform:.
To deal with those cases, I would suggest adding a simple annotation
that idc could treat as a unary identity pseudo-method that just means
the receiver is not an oop. The obvious name is "_", used like this:
"Make sure we catch any attempts to call the non-oop annotation as if
it were a regular method. Idc should remove all these sends."
_object _ [ ^self error: 'compiler error: stray non-oop annotation' ]
obj perform: sel with: aNonOop _ with: anOop
([ { return (oop)42; } ] value: a with: b) _
This is no more onerous than having to teach people the difference
between _self (the struct) and v_self (the abstract prototype) even
though they amount to the same thing in most cases.
Once that's done, I believe every variable in the whole COLA
implementation can be categorised as clearly oop or non-oop, without
exceptions.
Garbage Collection
******************
I have the beginnings of a design for a precise GC. The basic idea is
to put all the stack-allocated struct __sends in a special zone that
keeps track of the size of the structure, prepending it as metadata.
struct __send
{
oop _zone[0];
oop _vtable[0];
oop receiver;
oop state;
oop selector;
struct __closure *closure;
struct __send *parent;
struct t__thread *thread;
struct __methodinfo *info;
oop line;
/* Local variables... */
oop local_oop[0];
};
#define _send_out (&_send_out_struct.send)
struct {
oop size;
oop _zone;
oop _vtable;
struct __send send;
} _send_out_struct = {
sizeof(struct __send) << 1 | 1,
sized_pure_oop_zone,
_send_out_vtable,
{ [struct __send initializers]... }
};
Everything in struct __send will be an oop, so object tracing of a
struct __send can be extremely simple. Idc will allocate all
underscore-prepended variables as it does now (as regular C stack
variables), but will need some extra machinery to allocate other
"formerly C stack" variables on the tail of the struct __send. I
expect that idc would add a bunch of "#define v_MYVAR
_send_out->local_oop[NNN]" lines to each method.
Object tracing would be written as a method on _object (and _vector,
as a special exception due to its indexability), by using idc's slot
name underscore conventions to determine which members are not really
oops. As explained below, I would suggest that the trace method
invokes a callback block with the object address, slot address, and a
relocation object (which for objects that are not functions would
always be OopRelocation - the slot holds an oop).
Of course, this all gains us nothing over the Boehm GC unless we're
sure that plain-old-C code always registers any pointers to our heap
(a rare occurance), and we would also have to mark all the C oops as
volatile. With those precautions, there would never be any live oops
in registers (or spills) when garbage collection occurs.
Having a compiler flag to disable the precise-precautions would be
interesting as a way of testing the relative performance of GCs. As
with the multithreading features, the rest of this can be implemented
by memory zones with metadata, which would allow for alternative GCs
at runtime, given an interface to stop the world and look at the
thread objects to find their stacks (and/or latest struct __sends).
Jolt Executables
****************
Applying the fundamental idea I used in a Pliant
(http://fullpliant.org/) static compiler I wrote many years ago, I
would like also to modify Jolt to add relocation information to
individual functions. This would be to support the object trace
methods, with potentially any kind of linker relocation.
Using these methods, it is simple to write a program serialiser that,
when invoked on a function, just dumps out an assembly file
(something.s) that contains a whole lot of byte data directives and
some labels (the function's label is "main:", which is a global, but
everything else is local). The resulting file can be assembled to
produce an executable for that platform.
I'm not sure how close that lies to your plans for static Jolt
executables, but it's a quick-and-dirty way to get something that
starts a whole lot faster than JITting scripts every time from
scratch.
That would allow us to start experimenting with multiphase compilation
and techniques for late-bound vs. early-bound linking. Jolt already
has the late-bound arena covered, so I'm interested in helping stretch
it in the other direction.
Anyway, this was a long e-mail, but I hope some of these ideas spark a
few brains on this list to come up with better designs and/or
comments.
Thanks,
--
Michael FIG <[EMAIL PROTECTED]> //\
http://michael.fig.org/ \//
_______________________________________________
fonc mailing list
[email protected]
http://vpri.org/mailman/listinfo/fonc
