At Thu, 13 Jun 2019 18:16:50 -0700, Matthew Butterick wrote: > > > On Jun 13, 2019, at 10:04 AM, Matthew Flatt <mfl...@cs.utah.edu> wrote: > > > > I recommend a lock-serving place in Pollen to manage concurrency for > > parallel rendering. Don't let multiple places try to read and/or write > > to the same file, and don't try to use filesystem locks. > > > Thanks for the suggestion. I see what you mean. Locking the source file being > rendered did reduce the filesystem errors ... > > ... but it didn't eliminate them. The wrinkle is that in the typical > Pollen project, many (or even all) these source files may > transitively rely on writing the same template file, e.g., > "template.html.p". I could lock that file too, but then other renders > that rely on it would be blocked, which would tend to relinearize the > job.
The compilation part of `raco setup` recovers some parallelism by detecting "prefetch" messages that are generated by the `require` macro. The prefetch messages report modules that will be demanded soon via the module name resolver, so compilation can start on them in parallel. It doesn't recover the potential parallelism completely, but it helps. Generating and/or catching prefetch messages might be a good way to go in your case, too. > As an alternative to file locking, I tried having each rendering > place attempt the render three times with a short delay in between. > On the idea that a temporary filesystem error is likely to resolve by > the third try, and a permanent failure (e.g., a problem with the > source file itself) will not, and can be raised as a "real" error. > > Maybe there is a secretly perncious aspect to this muddle-through-it > technique, though it does avoid the relinearization problem. Oh, don't do that. Unreliable workarounds to concurrency problems really do come back to bite you later. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/5d02fbdb.1c69fb81.35914.1662SMTPIN_ADDED_MISSING%40gmr-mx.google.com. For more options, visit https://groups.google.com/d/optout.
