I am more interested in syntax two than syntax one, which I felt should
probably be included for completeness. But hey, as you say, maybe not since
unguarded concurrency is indeed usually a mistake.
Taken on its own, do you have an objection to `for (item of items
concurrency 5) { ... }`?
On Fri, Sep 6, 2019 at 1:46 PM C. Scott Ananian <ecmascr...@cscott.net>
wrote:
> The current way to write what you want is:
>
> await Promise.all(itemsCursor.map(item => db.insert(item));
>
> or
>
> await Promise.all(itemsCursor.map(Promise.guard(5, item =>
> db.insert(item))));
>
> if you are using a library like `prfun` (
> https://github.com/cscott/prfun#promiseguardfunctionnumber-condition-function-fn--function
> ).
>
> I'm not sure adding a new parallel loop construct is an obvious
> improvement on this, especially since unguarded concurrent execution is
> usually a mistake (can cause memory requirements to blow up), as you point
> out yourself.
>
> I'd be more in favor of new syntax if it was integrated with a
> work-stealing mechanism, since (a) that can't as easily be done in a
> library function (you need access to the entire set of runnable tasks, not
> just the ones created in this loop), and (b) is more likely to be
> correct/fast by default and not lead to subtle resource-exhaustion problems.
> --scott
>
> On Fri, Sep 6, 2019 at 12:40 PM Tom Boutell <t...@apostrophecms.com> wrote:
>
>> *Specifying concurrency for "for...of" loops potentially containing
>> "await" statements in the loop body*
>>
>> In the async/await era, I see most developers using the async and await
>> keywords in 90% of situations, shifting to "Promise.all" or the bluebird
>> library only to cope with concurrency issues.
>>
>> The most common case in my experience is the need to admit a manageable
>> level of parallelism when iterating a large array (or iterator, see the
>> final section) and performing asynchronous work on each item. Unlimited
>> concurrency (Promise.all) tends to overwhelm backends involved in a way
>> that confuses developers as to what is happening. Bluebird's "Promise.map"
>> permits concurrency to be specified, which is great, but requires pulling
>> in a library and switching of mental gears ("OK right, these async
>> functions return promises," etc).
>>
>> To build on the friendliness of async/await, I propose these two syntaxes
>> be accepted:
>>
>> SYNTAX ONE
>>
>> ```js
>> for (item of items concurrent) {
>> // db.insert is an async function
>> await db.insert(item);
>> }
>> ```
>>
>> In Syntax One, all loop bodies commence concurrently (see below for the
>> definition of "concurrently" with regard to async). If an exception is not
>> caught inside the loop, it is thrown beyond the loop, and all exceptions
>> subsequently thrown by concurrently executing loop bodies are discarded
>> (like Promise.all).
>>
>> *While I feel that unlimited concurrency is usually a mistake in a
>> situation where you have an array of items of unpredictable number, it
>> seems odd not to have a syntax for this case, and "concurrency 0" seems
>> clunky.*
>>
>> SYNTAX TWO
>>
>> ```js
>> for (item of items concurrency 5) {
>> // db.insert is an async function
>> await db.insert(item);
>> }
>> ```
>>
>> in Syntax Two, up to 5 loop bodies commence concurrently (see below).
>> There is no guarantee that item 3 will finish before item 2, or that item 4
>> won't start (due to 3 being finished) before item 2 ends, etc. If an
>> exception is not caught inside the loop, it is thrown beyond the loop, and
>> all exceptions subsequently thrown by concurrently executing loop bodies
>> are discarded (like Promise.all in this respect, except for the restriction
>> of concurrency).
>>
>> DEFINING CONCURRENCY FOR ASYNC
>>
>> For purposes of this proposal, "concurrent" execution means that multiple
>> loop bodies may be suspended via "await" at any given time. It does NOT
>> refer to multithreaded execution, worker threads, etc.
>>
>> CONSIDERATIONS FOR ASYNC ITERATORS
>>
>> Async iterator syntax for "for...of" loops, as in:
>>
>> ```js
>> for await (item of itemsCursor) { ... }
>> ```
>>
>> Should also support concurrency for the loop body, with the same syntax:
>>
>> ```js
>> for await (item of itemsCursor concurrency 5) { ... }
>> ```
>>
>> *It is important to note that this syntax does not add concurrency to the
>> async iterator itself, *at least not at this time, as I believe the
>> interface for defining async iterators does not currently accommodate this.
>> However this syntax is still useful because it *fetches the items
>> sequentially from the iterator, but may "fill the hopper" with up to five
>> iterator results* that are currently being actively processed by loop
>> bodies. In many cases, fetching items via an iterator is much faster than
>> the processing that will be done to them in the loop bodies, and so this is
>> still useful.
>>
>> Thanks for reading!
>>
>> --
>> Chief Software Architect
>> Apostrophe Technologies
>> Pronouns: he / him / his
>> _______________________________________________
>> es-discuss mailing list
>> es-discuss@mozilla.org
>> https://mail.mozilla.org/listinfo/es-discuss
>>
>
--
Chief Software Architect
Apostrophe Technologies
Pronouns: he / him / his
_______________________________________________
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss
