*Specifying concurrency for "for...of" loops potentially containing "await"
statements in the loop body*
In the async/await era, I see most developers using the async and await
keywords in 90% of situations, shifting to "Promise.all" or the bluebird
library only to cope with concurrency issues.
The most common case in my experience is the need to admit a manageable
level of parallelism when iterating a large array (or iterator, see the
final section) and performing asynchronous work on each item. Unlimited
concurrency (Promise.all) tends to overwhelm backends involved in a way
that confuses developers as to what is happening. Bluebird's "Promise.map"
permits concurrency to be specified, which is great, but requires pulling
in a library and switching of mental gears ("OK right, these async
functions return promises," etc).
To build on the friendliness of async/await, I propose these two syntaxes
be accepted:
SYNTAX ONE
```js
for (item of items concurrent) {
// db.insert is an async function
await db.insert(item);
}
```
In Syntax One, all loop bodies commence concurrently (see below for the
definition of "concurrently" with regard to async). If an exception is not
caught inside the loop, it is thrown beyond the loop, and all exceptions
subsequently thrown by concurrently executing loop bodies are discarded
(like Promise.all).
*While I feel that unlimited concurrency is usually a mistake in a
situation where you have an array of items of unpredictable number, it
seems odd not to have a syntax for this case, and "concurrency 0" seems
clunky.*
SYNTAX TWO
```js
for (item of items concurrency 5) {
// db.insert is an async function
await db.insert(item);
}
```
in Syntax Two, up to 5 loop bodies commence concurrently (see below). There
is no guarantee that item 3 will finish before item 2, or that item 4 won't
start (due to 3 being finished) before item 2 ends, etc. If an exception is
not caught inside the loop, it is thrown beyond the loop, and all
exceptions subsequently thrown by concurrently executing loop bodies are
discarded (like Promise.all in this respect, except for the restriction of
concurrency).
DEFINING CONCURRENCY FOR ASYNC
For purposes of this proposal, "concurrent" execution means that multiple
loop bodies may be suspended via "await" at any given time. It does NOT
refer to multithreaded execution, worker threads, etc.
CONSIDERATIONS FOR ASYNC ITERATORS
Async iterator syntax for "for...of" loops, as in:
```js
for await (item of itemsCursor) { ... }
```
Should also support concurrency for the loop body, with the same syntax:
```js
for await (item of itemsCursor concurrency 5) { ... }
```
*It is important to note that this syntax does not add concurrency to the
async iterator itself, *at least not at this time, as I believe the
interface for defining async iterators does not currently accommodate this.
However this syntax is still useful because it *fetches the items
sequentially from the iterator, but may "fill the hopper" with up to five
iterator results* that are currently being actively processed by loop
bodies. In many cases, fetching items via an iterator is much faster than
the processing that will be done to them in the loop bodies, and so this is
still useful.
Thanks for reading!
--
Chief Software Architect
Apostrophe Technologies
Pronouns: he / him / his
_______________________________________________
es-discuss mailing list
[email protected]
https://mail.mozilla.org/listinfo/es-discuss
