I think it would help to take a step back and think about what you're doing
when you communicate with a place. As you know, places are effectively
separate instances of the Racket VM: other than the explicit, low-level
mechanisms like `make-shared-bytes`, they share no state at all, not even
the same module instances. Let's imagine we have this module and have
required it in two different places:
(module example racket
  (provide remember!
           recall)
  (define store
    (make-hash))
  (define (remember! k v)
    (hash-set! store k v))
  (define (recall k)
    (hash-ref store k)))
Each place would have its own distinct instance of that module, each with
its own hash table: mutating the hash table in one place would not change
the other place's hash table.

This is the reason why procedures can't be sent across places. Place A's
version of `remember!` closes over Place A's version of `store`. It doesn't
know anything about Place B's version of `store`, so it certainly can't
mutate that. Allowing Place A's `remember!` to be called from Place B and
mutate Place A's version of `store` would violate the safety guarantees
that places provide by requiring explicit message-passing rather than
shared state.

That means, if you want one place to tell another to call a function, you
need to send it some kind of immutable message telling it what to do. It's
rather like calling an API over the network. Let's say you want to run tell
some place to execute the following thunk:
(λ ()
  (+ 1 2))

How might you represent that function as data?

Well, if a function is available as a module-level export, you can access
it with `dynamic-require`, so a natural way to represent "call this
function with these arguments" would be with a list of arguments for
`dynamic-require` to get the function you have in mind, plus a list of the
arguments to give to the function. The example above might be represented
like this:
'([racket/base +] . [1 2])

The receiver place could then interpret such a message like this:
(λ (message)
  (apply (apply dynamic-require (car message))
         (cdr message)))

That's essentially how `serial-lambda` works under the hood. Each syntactic
use of the `serial-lambda` macro is turned into a module-level structure
type definition that implements `prop:procedure`. When a use is evaluated
and a closure is allocated, it creates an instance of that structure type,
packaging up its free lexical variables into the structure's fields (that's
the hard part). The `prop:serializable` protocol for `racket/serialize`
essentially records the same information you would need to use
`dynamic-require`.

So, to answer some of your specific questions:

On Sun, Apr 15, 2018 at 10:51 AM, Zelphir Kaltstahl <
zelphirkaltst...@gmail.com> wrote:

> - What if in that serial-lambda the user needs to use some custom
> procedure? Does that suddenly also have to be serializable? What about
> its "dependencies"? --> everything in the user program ends up being a
> serial-lambda. That would be really bad.
>

For the procedure value to be serializable, all of the values it lexical
closes over have to be serializable. If you remember that those values have
to be packaged up into fields of a struct, this makes sense: a list is also
only serializable if its contents are serializable.

It takes some experience to readily recognize just what it is that an
anonymous function will close over. One helpful rule is that module-level
variables are never part of the closure, so they aren't required to be
serializable: thus, it's ok that things like + aren't serializable. On the
other hand, in this example:
(define (make-thunk x)
  (serial-lambda ()
    (println x)))
the function returned by make-thunk will only be serializable when `x` is
serializable.

You can find some more background about this in the #lang web-server
documentation. I also wrote some notes on serialization pitfalls for the
`web-server/formlets` library, which (now) uses serializable procedures
internally: http://docs.racket-lang.org/web-server/formlets.html#%
28part._.Formlets_and_.Stateless_.Servlets%29


> - Is there a better way than requiring everything to be serial-lambda?
>

With the caveat that, as I said, not "everything" has to use serial-lambda,
I don't think there is a better way. Any other solution for serializing an
arbitrary function would just end up re-implementing what
`web-server/lang/serial-lambda` does (and has been tested and used in
production doing). I can think of things that `serial-lambda` doesn't
do—for example, I've experimented with trying to find a mechanism for
serialized procedures to take their contracts with them—but I would want to
use `serial-lambda` to implement such additional features, not replace
`serial-lambda`. It does its job very well.


> - Is the idea to have lambdas be serializable by default language wide
> insane? It would be great to be able to simply start a new place and
> give it some arbitrary lambda to execute.
>

#lang web-server/base is just like #lang racket/base, except newly created
functions are serializable by default (as are continuations). However,
there is overhead in making a function serializable, and it probably
wouldn't be a good default for the overwhelming majority of functions that
nobody ever wants to serialize.

Finally, on a broader point, I don't think you can avoid having to think
about the fact that your code is going to run in parallel: for example, you
will always have to make sure that you don't depend on shared state. With
the implementation of places in particular, you also need to consider
communication overhead and startup time. If I were designing a library for
parallelism, I would start with some specific use-case in mind and focus on
coming up reusable solutions for specific sub-parts of the problem.

Hope some of this helps.

-Philip

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to