RFC: Implicit threading and Implicit event-loop (Was: Re: Continuations)

2009-05-27 Thread Daniel Ruoso
Em Ter, 2009-05-26 às 19:33 -0700, Jon Lang escreveu:
 The exact semantics of autothreading with respect to control
 structures are subject to change over time; it is therefore erroneous
 to pass junctions to any control construct that is not implemented via
 as a normal single or multi dispatch. In particular, threading
 junctions through conditionals correctly could involve continuations,
 which are almost but not quite mandated in Perl 6.0.0.
 What is a continuation?

Continuation here is meant in the most generic sense, which is:

The rest of the thread of execution

It doesn't imply any specific API on manipulating the continuations, nor
it implies that the continuations are re-invocable, cloneable or
anything like that.

It basically means that the interpreter can choose to interrupt your
code at any point and continue it later, after running some other code.
This has the basic effect that Perl 6 points toward *implicit threading*
rather than explicit, and also that it points toward *implicit event
loop* rather than explicit.

In practical terms:

 sub baz (*...@input) {
   for @input - $element {
 say BAZ!;
 $element + 1;
   }
 } 

 sub foo (*...@input) {
   for @input - $element {
 say FOO!;
 $element - 1;
   }
 }

 sub bar (*...@input) {
   for @input - $element {
 say BAR!;
 $element * 2;
   }
 }

 say BEFORE!;
 my @a == baz == foo == bar == $*IN;
 say AFTER;

Is going to open 5 implicit threads (which might be delegated to any
number of worker threads), besides the initial thread. So, at first,
you'll immediatly see in the output:

 BEFORE!
 AFTER!

The implicit threads are:

  1 - read from $*IN and push into a lazy list X
  2 - read from the lazy list X, run an iteration of the for in the sub 
  bar, and push to the lazy list Y
  3 - read from the lazy list Y, run an iteration of the for in the sub
  foo, and push to the lazy list W
  4 - read from the lazy list W, run an iteration of the for in the sub
  baz, and push to the lazy list Z
  5 - read from the lazy list Z and push into the lazy list that
  happens to be stored in '@a'

That basically means that this lazy lists are attached to the
interpreter main-loop (yes, Perl 6 should implement something POE-like
in its core), which will allow the read of IO to be non-blocking, so you
don't need a OS thread for that. It also means that every lazy list
should be somehow attached to that event-loop.

So, as you enter data in $*IN, you should get something like that:

 I entered this line!
 BAR!
 FOO!
 BAZ!
 I entered this other line!
 BAR!
 FOO!
 BAZ!

On the implementation side, I think there is going to be a
ControlExceptionWouldBlock, which is raised by every lazy object when
the data is not immediatly available, allowing the interpreter to put
this continuation in a blocked state, somehow registering a listener
to the event that blocks it.

One of the attributes of the ControlExceptionWouldBlock would be a
Observable object, this Observable object is the thing that is waiting
for the specific event to happen and register additional listeners to
that event. The interpreter itself will register itself as an Observer
to that Observable, so it can re-schedule the thread, marking it as
waiting.

That being said, I think we have a continuation pool which are in
either running, blocked or waiting state. And a scheduler that
takes this continuations and assign to the worker threads, while you
can use a command line switch to control the minimum/maximum number of
worker threads as well as the parameter for when to start a new worker
thread and when to deactivate it...

Well, this is my current view on the state of affairs, and is thougth a
lot in the context of SMOP, so it would be really interesting to have
some feedback from the parrot folks...

daniel



Re: RFC: Implicit threading and Implicit event-loop (Was: Re: Continuations)

2009-05-27 Thread John M. Dlugosz

Sounds like threads to me.

What I see that's different from common threads in other languages is 
that they are all the same, rather than one master and many new threads 
that have no context history above them.  In Perl 6, every thread sees 
the same dynamic scope as the original.  It doesn't matter which one is 
left standing to continue and eventually return up the context chain. 


--John


Daniel Ruoso daniel-at-ruoso.com |Perl 6| wrote:

Em Ter, 2009-05-26 às 19:33 -0700, Jon Lang escreveu:
  

The exact semantics of autothreading with respect to control
structures are subject to change over time; it is therefore erroneous
to pass junctions to any control construct that is not implemented via
as a normal single or multi dispatch. In particular, threading
junctions through conditionals correctly could involve continuations,
which are almost but not quite mandated in Perl 6.0.0.
What is a continuation?



Continuation here is meant in the most generic sense, which is:

The rest of the thread of execution

It doesn't imply any specific API on manipulating the continuations, nor
it implies that the continuations are re-invocable, cloneable or
anything like that.

It basically means that the interpreter can choose to interrupt your
code at any point and continue it later, after running some other code.
This has the basic effect that Perl 6 points toward *implicit threading*
rather than explicit, and also that it points toward *implicit event
loop* rather than explicit.

In practical terms:

 sub baz (*...@input) {
   for @input - $element {
 say BAZ!;
 $element + 1;
   }
 } 


 sub foo (*...@input) {
   for @input - $element {
 say FOO!;
 $element - 1;
   }
 }

 sub bar (*...@input) {
   for @input - $element {
 say BAR!;
 $element * 2;
   }
 }

 say BEFORE!;
 my @a == baz == foo == bar == $*IN;
 say AFTER;

Is going to open 5 implicit threads (which might be delegated to any
number of worker threads), besides the initial thread. So, at first,
you'll immediatly see in the output:

 BEFORE!
 AFTER!

The implicit threads are:

  1 - read from $*IN and push into a lazy list X
  2 - read from the lazy list X, run an iteration of the for in the sub 
  bar, and push to the lazy list Y

  3 - read from the lazy list Y, run an iteration of the for in the sub
  foo, and push to the lazy list W
  4 - read from the lazy list W, run an iteration of the for in the sub
  baz, and push to the lazy list Z
  5 - read from the lazy list Z and push into the lazy list that
  happens to be stored in '@a'

That basically means that this lazy lists are attached to the
interpreter main-loop (yes, Perl 6 should implement something POE-like
in its core), which will allow the read of IO to be non-blocking, so you
don't need a OS thread for that. It also means that every lazy list
should be somehow attached to that event-loop.

So, as you enter data in $*IN, you should get something like that:

 I entered this line!
 BAR!
 FOO!
 BAZ!
 I entered this other line!
 BAR!
 FOO!
 BAZ!

On the implementation side, I think there is going to be a
ControlExceptionWouldBlock, which is raised by every lazy object when
the data is not immediatly available, allowing the interpreter to put
this continuation in a blocked state, somehow registering a listener
to the event that blocks it.

One of the attributes of the ControlExceptionWouldBlock would be a
Observable object, this Observable object is the thing that is waiting
for the specific event to happen and register additional listeners to
that event. The interpreter itself will register itself as an Observer
to that Observable, so it can re-schedule the thread, marking it as
waiting.

That being said, I think we have a continuation pool which are in
either running, blocked or waiting state. And a scheduler that
takes this continuations and assign to the worker threads, while you
can use a command line switch to control the minimum/maximum number of
worker threads as well as the parameter for when to start a new worker
thread and when to deactivate it...

Well, this is my current view on the state of affairs, and is thougth a
lot in the context of SMOP, so it would be really interesting to have
some feedback from the parrot folks...

daniel