On Wed, Oct 9, 2019 at 2:09 PM Zelphir Kaltstahl <zelphirkaltst...@gmail.com>
wrote:
> I was wrongly under the impression, that serializable-lambda are supposed
> to work out of the box, when sending them over channels, without needing to
> do any further work ("are serialized automatically" instead of "can be
> serialized"). … If you know how to serialize those serializable-lambdas, it
> is possible, that you could solve the problems they faced.
>
> … Is there a generic way to serialize such lambdas, no matter what they
> look like?
>
You can serialize the procedures produced by `serial-lambda` using the `
racket/serialize <https://docs.racket-lang.org/reference/serialization.html>`
library, the same way you serialize other kinds of Racket values. Here is
an extended example:
#lang racket
(require web-server/lang/serial-lambda
racket/serialize
rackunit)
(define (make-serializable-adder n)
(serial-lambda (x)
(+ n x)))
(define serializable-add5
(make-serializable-adder 5))
;; The value of a serial-lambda expression is a procedure.
(check-eqv? (serializable-add5 2)
7)
;; The procedure can't be sent over a place-channel directly ...
(check-false
(place-message-allowed? serializable-add5))
;; ... but it is serializable:
(check-true
(serializable? serializable-add5))
(define serialized
(serialize serializable-add5))
;; The serialized form can be sent over a place-channel.
(check-true
(place-message-allowed? serialized))
(define-values [in out]
(place-channel))
(place-channel-put in serialized)
;; When we deserialize the received value, potentially in a new place ...
(define received
(place-channel-get out))
(define deserialized
(deserialize received))
;; ... it works like the original, including closing over the lexical
environment.
(check-eqv? (deserialized 11)
16)
One thing to note is that the procedures returned by `serial-lambda` do
close over their lexical environments, which is desirable for the reasons
you mention in your previous message. That means that values which are
captured as part of the closure must also be serializable, or `serialize`
will raise an exception. (The same is true for lists: they are serializable
as long as their contents are also serializable.)
Concretely, in the example above, `n` is part of the closure and therefore
must be serialized—which works out great, because `n` must be a number, and
numbers are serializable. On the other hand, the procedure `+` isn't
serializable, but that's ok, because `+` isn't part of the closure: it's a
reference to a module-level identifier.
… it seems to be the case in other programming languages, where one can
> simply send lambdas to other actors, which can, but don't have to, run on
> other cores or other machines. … It also seems to be necessary for
> eventually creating a library, which provides a process pool, as such
> library possibly should be easy to use and not force the user to think
> about difficult thing, when the intention the user has seems so simple
> "just do that on another core".
>
This depends on what you mean by "actors." You can simply send closures to
other Racket-level threads (in the sense of `thread
<https://docs.racket-lang.org/reference/threads.html>`), which share state
and run concurrently (but not in parallel) within the same OS-level thread.
While not framed specifically in the vocabulary of the actor model, Racket
threads seem to fit the model pretty well: they are what I think of first
when I hear "actors" in a Racket context.
When you want machine-level parallelism (i.e. multiple OS threads), that
shared mutable state becomes a problem, because hardware provides only very
low-level mechanisms for coordinating parallel threads so that they have a
consistent view of the state (i.e. no variables are half-assigned). This is
true regardless of the language you work in: languages differ in what
tools, if any, they give you to manage the problem.
Racket provides two mechanisms for parallelism, places and futures
<https://docs.racket-lang.org/reference/futures.html>. Futures share state
like Racket-level threads, but block when attempting an operation that
can't safely be done in parallel: in other words, they provide
"best-effort" parallelism. On traditional Racket, there are lots of
operations that can't safely be done in parallel, so futures have mostly
been useful so far for numeric computation. There's hope that will change
with Racket-on-Chez, but I think we're still at the stage of needing more
people to try it and see how it goes in practice.
Places, on the other hand, provide "shared nothing" parallelism: each
exists in its own world, without shared mutable state, and they communicate
by explicitly sending messages over place-channels (which are different
than normal channels, e.g. by being asynchronous, but have much the same
API and integrate with Racket's general system for threads and events).
(Actually, places do support a very restricted form of shared mutable state
through things like `make-shared-bytes`.) There are also distributed places
and Paulo Matos's loci <https://github.com/LinkiTools/racket-loci>, which
extend the idea of places to multiple processes or machines.
The point is that "just do that on another core" doesn't work in general,
because "that" might rely on mutable state or other aspects of the context
where it is run. Of course there are lots of cases that are trivially
parallelizable, but, even in those cases, starting parallel threads and
communicating between them involves overhead. I don't know of any language
or library that completely relieves the programmer of thinking about what
can be parallelized well and how much parallelism to use, and then writing
those decisions down somehow in the programming language. In my experience,
Racket makes it easier to reason about parallel programming than languages
that expose you to the wild west of the hardware, and I don't find writing
down parallel programs in Racket especially onerous.
-Philip
--
You received this message because you are subscribed to the Google Groups
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/racket-users/CAH3z3gZp2rEEwEzPMHt0be_vU%2BEDYY7%3D8wQxCrchWcSGdsX6BQ%40mail.gmail.com.
