Hi Leszek,
On Fri, Sep 12, 2025 at 2:18 AM Leszek Swirski <[email protected]> wrote:
> We took a closer look at how this works in V8+Chromium, and long story
> short we only have one microtask queue shared between different contexts
> that can interact with each other. I think this is the only real reasonable
> solution, if I'm honest, multiple microtask queues interleaving with each
> other are asking for trouble. If the node module has any dependencies on
> promises from the outer context, I expect you'll hit the same issue in
> reverse, and I bet that you could interleave chaining promises from inside
> and outside to cause this problem up to an arbitrary depth. Could the inner
> source module post a full event loop task after it finishes its execution,
> and resolve the execute promise in that? Then any microtasks started by the
> execute are guaranteed to complete before that task executes.
>
Yes, I agree, the interactions between contexts can get really complicated,
with promises shared back and forth, or transitively across N contexts with
N queues.
The following example is indeed broken, even with our tentative fix in
node:vm. Waiting on `outer_promise` lets the execution flow fall through.
```js
import * as vm from "node:vm";
const context = vm.createContext({
console,
outer_promise: Promise.resolve(),
}, {
microtaskMode: "afterEvaluate",
});
const m = new vm.SourceTextModule(
// OUPS, we enqueued a task on the global queue:
'await outer_promise;',
{context},
);
await m.link(() => null);
await m.evaluate();
// The global queue will be drained, but we would then need to drain the
// inner queue again, and we've already given up on our chance to do that.
console.log("NOT PRINTED");
```
I now wonder a little bit if the standards folks threw in the concept of
multiple microtask queues without a clear use case and it's not really
workable as it is.
The tentative fix in Node, an idea of @addaleax, is narrow and aligns with
what you're suggesting I think: instead of returning a Promise built in the
inner context from `module.evaluate()`, we return a promise built in the
outer context. This is done within the node:vm library, by resolving an
outer Promise with the return value of v8::SourceTextModule::Evaluate(),
which was built in the inner context, and then we checkpoint the inner
context microtask queue once. This at least makes it possible to write:
```
await module.evaluate();
```
without blowing up the execution flow. The `module` object is meant to
mimic the SourceTextModule() API, so returning an outer-context Promise is
a slight departure from the API, but it is necessary to make it actually
usable with a separate queue.
BTW, the use of multiple queues in node:vm is optional; it is needed when
the user wants to constrain the inner module with a timeout or SIGINT. I'm
not sure there is a way to implement a timeout feature for the inner
context without a separate queue.
We're considering adding a section to the node:vm documentation warning
users of the (remaining) consequences of sharing promises between different
contexts, when using multiple microtask queues. It's not pretty, see the
end of this email. I'm not even sure that this new section is scary enough!
With that in mind, I still think it might be a good idea for v8 to offer a
notification mechanism on MicrotaskQueue. I do get your point:
foreign-context enqueuing can happen in any direction between contexts.
Since the global queue uses kAuto, however, only the inner queues need to
be "monitored". If I simulate such a notification mechanism with a
setInterval() in the example at the top of this email, the code can be made
to behave sanely.
Right now, while user-code can try to anticipate the need to manually drain
the inner queue, by setting up a callback with setInterval(), say, it is
not clear when it is safe to *stop* doing so. You'd need to detect that
you've made enough progress both inside the inner module and within the
global context, and call clearInterval(). It can get complicated. There is
also no way to simply *detect* that you've hit this strange behavior.
```js
module.onPendingExternalMicrotasks(() => {
// Here can be added timeout logic, or simply print a warning that some
unexpected
// promise sharing has happened.
module.evaluate();
});
```
Thanks,
Eric
### When `microtaskMode` is `'afterEvaluate'`, beware sharing Promises
between Contexts
In `'afterEvaluate'` mode, the `Context` has its own microtask queue,
separate
from the global microtask queue used by the outer (main) context. While this
mode is necessary to enforce `timeout` and enable `breakOnSigint` with
asynchronous tasks, it also makes sharing promises between contexts
challenging.
In the example below, a promise is created in the inner context and shared
with
the outer context. When the outer context `await` on the promise, the
execution
flow of the outer context is disrupted in a surprising way: the log
statement
is never executed.
```mjs
import * as vm from 'node:vm';
const inner_context = vm.createContext({}, { microtaskMode: 'afterEvaluate'
});
// runInContext() returns a Promise created in the inner context.
const inner_promise = vm.runInContext(
'Promise.resolve()',
context,
);
// As part of performing `await`, the JavaScript runtime must enqueue a task
// on the microtask queue of the context where `inner_promise` was created.
// A task is added on the inner microtask queue, but **it will not be run
// automatically**: this task will remain pending indefinitely.
//
// Since the outer microtask queue is empty, execution in the outer module
// falls through, and the log statement below is never executed.
await inner_promise;
console.log('this will NOT be printed');
```
To successfully share promises between contexts with different microtask
queues,
it is necessary to ensure that tasks on the inner microtask queue will be
run
**whenever** the outer context enqueues a task on the inner microtask queue.
The tasks on the microtask queue of a given context are run whenever
`runInContext()` or `SourceTextModule.evaluate()` are invoked on a script or
module using this context. In our example, the normal execution flow can be
restored by scheduling a second call to `runInContext()` _before_ `await
inner_promise`.
```mjs
// Schedule `runInContext()` to manually drain the inner context microtask
// queue; it will run after the `await` statement below.
setImmediate(() => {
vm.runInContext('', context);
});
await inner_promise;
console.log('OK');
```
--
--
v8-dev mailing list
[email protected]
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups
"v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/v8-dev/CA%2BzRj8UjT7v%2BywhTHBzjbVGs%2BSXtGY5C%2Bome%3DBX408fFZS2DjA%40mail.gmail.com.
