Below I have re-worked the example program you sent yesterday to send all the work to an explicitly-created place-channel, from which all of the worker places can get work.
I also saw your question about sending procedures to places. The module web-server/lang/serial-lambda provides the serial-lambda form, but creates procedures that can be serialized using racket/serialize (as long as the contents of the closure are serializable): http://docs.racket-lang.org/web-server-internal/closure.html Once serialized, these can be passed through place-channels, and they are far more robust than using eval. In general, the documentation for place-message-allowed? specifies what kind of messages can be passed through place-channels: http://docs.racket-lang.org/reference/places.html?q=place-message-allowed%3F#%28def._%28%28lib._racket%2Fplace..rkt%29._place-message-allowed~3f%29%29 #lang racket (define (place-output out-ch . args) (place-channel-put out-ch args)) (define (make-out-ch) (define-values (in-ch out-ch) (place-channel)) (thread (λ () (let loop () (apply printf (sync in-ch)) (newline) (loop)))) out-ch) (define (start-places get-work-pch out-ch) (for/list ([place-id (processor-count)]) (define place (place/context terminate-pch (place-output out-ch "place starting") (let loop () (sync (handle-evt terminate-pch (λ (_) (place-output out-ch "Place ~s is going to finish now." place-id))) (handle-evt get-work-pch (λ (data) (place-output out-ch "Place ~s got the following work: ~a." place-id data) (place-channel-put get-work-pch place-id) (loop))))))) (place-output out-ch "created place ~s" place-id) place)) (define (stop-places places) ;; simply terminate all places (for ([a-place (in-list places)]) (place-channel-put a-place 'terminate)) (for ([a-place (in-list places)]) (place-wait a-place))) (define (main [messages '(the quick brown fox jumped over the lazy dogs)]) (define-values (send-work-pch get-work-pch) ;; this must not be at the module level, or each place will ;; get its own version (place-channel)) (define out-ch (make-out-ch)) (define places (start-places get-work-pch out-ch)) (sleep 1) (define num-messages (for/fold ([n 0]) ([message messages]) (place-channel-put send-work-pch message) (add1 n))) (place-output out-ch "main has no more work data") (let loop ([done 0]) (unless (= done num-messages) (sync send-work-pch) (loop (add1 done)))) (place-output out-ch "all work reports finished") (stop-places places)) (module* main #f (main)) -Philip On Sun, Jan 14, 2018 at 8:18 AM, Zelphir Kaltstahl < [email protected]> wrote: > I also found the following post on StackOverflow: > https://stackoverflow.com/a/19095650/1829329 > > There the following timeout using construct is used: > > ~~~ > > (let loop ((i 0) (res '())) ; response loop > (if (= i l) > (reverse res) > (let ((response (sync/timeout 10 wch-c))) ; get answer with timeout > (place-channel-get blocks!) > (loop > (+ i 1) > (if response (cons response res) res)))))) > > ~~~ > > But it is not perfect, because the timeout time is wasted and could > already be used to distribute more work. > > > On Sunday, January 14, 2018 at 3:15:23 PM UTC+1, Zelphir Kaltstahl wrote: >> >> I have multiple places on which I would like to distribute some "work". I >> don't have a clear idea yet what that might be, so it is currently >> something abstract, which I don't want to make many assumptions about. >> Preferably I would like it to be so generic, that it could be anything. >> This is conceptually a "pool of places" I think. Such a pool of places is >> nice to have, because one does not always know the best way to parallelize >> things, unless it is a very simple case, where computation of all work >> items takes the same amount of time. With a pool which distributes the work >> itself, always giving the idle places work to do, it is like filling >> multiple containers with sand, rather than big stones which would leave >> gaps. >> >> I would like to "listen" on the channels of the places, until there is a >> message and then react on the message. >> The docs >> <https://docs.racket-lang.org/reference/places.html?q=place*#%28def._%28%28lib._racket%2Fplace..rkt%29._place-channel-get%29%29> >> say that `place-channel-get` is blocking. I also found the asynchronous >> buffered channels docs >> <https://docs.racket-lang.org/reference/async-channel.html>, but do not >> know if or how I could use them with places, or whether they are only for >> threads. >> The place-channel-get being blocking means that if I use that my work >> distribution would be blocked until I get some message from that place on >> its channel. >> The progress of computation would depend on that one checked channel >> giving me a message. >> >> I was thinking about putting such checks in threads, one for each >> channel, and then re-creating any thread that returns with some message >> from its place to send more work and get more messages, except if the work >> has all been distributed or the place has terminated. >> Somehow I would need to let the threads call a procedure to let them >> "signal", that they returned. >> Maybe 1 procedure for getting more work on the place and 1 procedure for >> signaling termination of the place. >> Otherwise I'd be doing busy wait again checking the threads for results, >> as I would have checked the channels without threads. >> >> But then I probably run into the next problem: How to manage the total >> available amount of work in a thread-safe manner? >> >> Is this even a feasible approach? >> Has there been work done that enables arbitrary computation (determined >> at run time, not compile time) in places distributing work to the places >> whenever they are no longer working on something? >> >> >> > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
