On 6/13/2019 9:16 PM, Matthew Butterick wrote:
On Jun 13, 2019, at 10:04 AM, Matthew Flatt <mfl...@cs.utah.edu
<mailto:mfl...@cs.utah.edu>> wrote:
I recommend a lock-serving place in Pollen to manage concurrency for
parallel rendering. Don't let multiple places try to read and/or write
to the same file, and don't try to use filesystem locks.
Thanks for the suggestion. I see what you mean. Locking the source
file being rendered did reduce the filesystem errors ...
... but it didn't eliminate them. The wrinkle is that in the typical
Pollen project, many (or even all) these source files may transitively
rely on writing the same template file, e.g., "template.html.p". I
could lock that file too, but then other renders that rely on it would
be blocked, which would tend to relinearize the job.
I don't know Pollen or your program architecture, so apologies if this
is a dumb suggestion.
I assume there is no required ordering to the "blocks" that you are
trying to render in parallel - i.e. it doesn't matter in what order the
output gets written to the template so long as each output block is
written contiguously (not interspersed with another).
If so, it would seem that each worker place could write into its own
temporary output file, and when finished pass the file back to the
dispatcher (task master) and let it assemble the final output. My
understanding is that places can pass file descriptors within the same
system, so although the dispatcher would have to copy finished work into
the template, the rendered content itself would not need to be passed
(piped,messaged), and since there would be just a single writer (the
dispatcher), you wouldn't need to lock the output template file(s).
YMMV,
George
--
You received this message because you are subscribed to the Google Groups "Racket
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to racket-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/racket-users/0ecd411a-2ebd-9d5c-1a92-d07dbf443a30%40comcast.net.
For more options, visit https://groups.google.com/d/optout.
