On Sat, 2006-01-07 at 13:38 +0100, Nicolas Cannasse wrote:
> > Unfortunately, any system .. including Neko and any other
> > VM .. which wishes to allow extensions to the API which call
> > C, and which also wishes to allow callbacks into the VM ..
> > is affected by this problem. The C stack gets in the way
> > of neutral control transfer. So even if it is implemented
> > in the Neko VM .. it will never work properly with C extensions
> > that need to callback into the VM.
>
> What I was thinking is using the current exceptions implementation but
> with a special form using $callc/$yield.
>
> f = function(delta) {
> var x = 0;
> while true {
> x += delta;
> $yield(x);
> }
> }
>
> $callcc(f,1);
>
> The $callcc is setting the "return point" of the continuation and is
> entering it. $yield will throw an exception up to the $callcc point.
> Like this it is some kind of parallel exception mecanism.
>
> The big difference is that $callcc will return the yield'ed value AND
> the continuation of f.
Yes, but where is that continuation put? Python iterators have a place
to put it .. your syntax so far doesn't. To extend your example
your proposing something like:
retval, cont = callcc(f,1);
where retval is the value returned by the yield, and cont is
the continuation which will resume execution after the yield.
Sort of right? So you could then make some decision, and then
either resume:
retval, cont = callcc(cont);
or drop the iteration and keep going. This is close to what
Felix does, except there is never any return value -- only
procedures use continuations, functions can't, mainly because
they're C compatible.
> It means that the VM stack between call/cc and
> yield is captured and returned as a new function that can be called
> again using either $callcc or just a normal call (in that case the
> $yield will go through it to the previous $callcc).
>
> The only problem here as you point out is that it doesn't play well with
> the C stack if there is callbacks, unless maybe you save it at the same
> time :
That is very hard to do right. It is likely to screw up Boehm_gc,
and C++ exception handling, and also old versions of pthreads (written
by Xavier Leroy in fact) which use the stack to identify which
pthread is running (and thus don't permit end user stack swapping).
[This issue arose on a Debian list recently .. Xavier might know
more .. I think it only affects old Linux kernels]
There are actually some functions for stack swapping,
setcontext and getcontext, in <ucontext.h> on Linux and
SysV (conforming to SUSv2 according to my man page).
Note also I am HIGHLY suspicious of setjmp/longjump,
I have many doubts about its workability in the presence
of C++ and other stuff. It's unlikely to work with
continuations -- if you 'jump' up the stack when you
yield the stack is gone .. it can't be restored.
Same goes for ANY code that fiddles the C stack.
too many functions -- eg alloca, sbrk, etc etc
may rely on the stack not changing. Very tricky.
> - save the vm stack
> - save the C stack
> - save a setjmp buffer
>
> then when calling back:
> - push back the vm stack
> - push back the C stack
> - longjmp to the yield C function
>
> That might be quite tricky to implement.
Yes indeed. Lua has this problem also, in its coroutines.
Felix doesn't .. it doesn't allow context switching except from
top level procedures. That's a heavy constraint, but the price
to be paid for using the C/C++ object model ;(
It also can't garbage collect except when the system
yields for a similar reason -- it uses an exact collector which
cannot know the layout of the C stack.
[Actually there is a nasty hack .. you can actually use
a C++ throw to throw a continuation .. the driver loop
catches it and replaces the current continuation with it ..]
Neko need not have such a heavy constraint: I think you can
abandon setjump/longjmp and the C stack entirely if you implement
functions with continuation passing. That is, your VM can just
ignore the C stack, so callbacks will just work fine.
This is actually quite easy to implement -- bending your head
around it is harder. Basically you just don't call subroutines
that aren't atomic primitives, instead you return (yield).
I think that means you cannot yield out of a callback,
instead you have to return a code that the loop driving
the callback -- in a GUI or whatever -- yields. This is
going to be very tricky, you're relying on the GUI having
the right 'idle hooks'. This is not a problem on MSWindows
by the way -- there is no 'run' function. Windows requires
you to read the event queue and dispatch the event manually:
the end user has to write the event loop. So it should be
easy to integrate it with a continuation passing model ..
but I haven't tried it myself yet :)
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
---
Neko : One VM to run them all
