On 01/28/2014 06:03 PM, Carter Schonwald wrote:
Theres actually a missing piece of information in this thread: what
are the example computations that are being sent? My understanding is
that erlang has not way to send file handles, shared variables,
Tvars, Mvars, memory mapped binary files, GPU code / memory pointers
, and other fun unportable things between nodes, and I don't really
expect / see how we can hope to sanely do that in haskell!
[...]"exactly the same binary, running on a cluster of homogeneous
machines with the exact same hardware, with a modern linux distro "
[...]
Nathan Howell was actually doing some experimentation with one
strategy for this special case here
https://github.com/alphaHeavy/vacuum-tube as a deeply rts twiddling
bit of hackery so you could in fact "serialize arbitrary closures"
between homogeneous machines running the exact same code (and with
address randomization disabled too i think)
When mentioning Nathan's approach (based on foreign primops), let me
point to a more complete, RTS-backed implementation; work done by myself
and itself based on a long-standing runtime support for a parallel
Haskell on distributed memory systems.
The latest instance of this rts-based serialisation was reported in the
Haskell-implementors' workshop 2013 (
www.haskell.org/wikiupload/2/28/HIW2013PackingAPI.pdf ); code is on
github (https://github.com/jberthold/rts-serialisation)
Some technical remarks:
-Nathan's prim.op approach is awesome, but it is not easy to get its
interplay with garbage collection right. It is on my list to take a look
at this code again and see how far we can push the envelope.
-About address randomisation: The RTS-based serialisation uses relative
locations from a known offset to handle it. A more concerning detail is
that CAFs must be reverted rather than discarded during GC (currently
they are just retained, not satisfactory for long-running code).
-About sending arbitrary closures: indeed it does not make any sense to
transfer MVars and IORefs (file handles, StablePtrs, etc). My approach
is to solve this dynamically by exception handling. I can imagine that
there is a sensible combination of RTS support with a suitable type
class framework (Static, for one), but lazy evaluation, especially lazy
I/O, complicates matters.
/ Jost Berthold
_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
