On Wed, Oct 13, 2010 at 04:00:02AM +0200, Leon Timmermans wrote:
> On Wed, Oct 13, 2010 at 12:46 AM, Tim Bunce <tim.bu...@pobox.com> wrote:
> > So I'd like to use this sub-thread to try to identify when lessons we
> > can learn from ithreads. My initial thoughts are:
> > - Don't clone a live interpreter.
> > Start a new thread with a fresh interpreter.
> > - Don't try to share mutable data or data structures.
> > Use message passing and serialization.
> Actually, that sounds *exactly* like what I have been trying to
> implementing for perl 5 based on ithreads (threads::lite, it's still
> in a fairly early state though). My experience with it so far taught
> me that:
> * Serialization must be cheap for this to scale. For threads::lite
> this turns out to be the main performance bottleneck. Erlang gets away
> with this because it's purely functional and thus doesn't need to
> serialize between local threads, maybe we could do something similar
> with immutable objects. Here micro-optimizations are going to pay off.
Being able to optionally define objects as structures in contiguous
memory could be a useful optimization. Both for serialization and
general cache-friendly cpu performance. Just a thought.
> * Code sharing is actually quite nice. Loading Moose separately in a
> hundred threads is not. This is not trivial though, Perl being so
> dynamic. I suspect this is not possible without running into the same
> issues as ithreads does.
If you wanted to start a hundred threads in a language that has good
support for async constructs you're almost certainly using the wrong
approach. In the world of perl6 I expect threads to be used rarely and
for specific unavoidably-bocking tasks, like db access, and where true
concurrency is needed.
Also, it should be possible to share read-only bytecode and perhaps
read-only jit'd executable pages, to avoid the full cost of "reloading"
> * Creating a thread (/interpreter) should be as cheap as possible,
> both in CPU-time as in memory. Creating an ithread is relatively
> expensive, specially memorywise. You can't realistically create a very
> large number of them the way you can in Erlang.
Erlang has just the one kind of concurrency mechanism: the "thread"
so you need to create lots of them (which Erlang makes very cheap).
We're looking at two concurrency mechanisms for perl6: Heavy-weight
O/S thread+interpreter pairs (as above), and lightweight async
behaviours within a single interpreter (eg continuations/fibers).
Those lightweight mechanisms are most like Erlang "threads". They'll be
cheap and plentiful, so they'll be far less need to start O/S threads.
> (well actually I learned a lot more; like about non-deterministic unit
> tests and profilers that don't like threads, but that's an entirely
> different story)
(Adding thread/multiplicity support to NYTProf shouldn't be too hard.
I don't have the time/inclination to do it at the moment, but I'll fully
support anyone who has.)