Re: es-discuss Digest, Vol 161, Issue 2

[Jan Krems]

The current answer is in the current spec for HostResolveImportedModule

:

> Each time this operation is called with a specific
referencingScriptOrModule, specifier pair as arguments
> it must return the same Module Record instance if it completes normally.

In other words: The runtime generally cannot safely collect modules because
all modules are "always" referenced
by the thing that imported it. It gets more strict in HTML where all
modules are globally referenced in the "module
map". So even when it *looks* like a module isn't referenced by previous
kinds of references (object properties etc.),
there really isn't such a thing as an "unreferenced" module as long as the
realm/context is alive.

In short: There's no "module cache" in ES modules, it's an append-only
module map.

This also means that if you expect something to require GC, you maybe
shouldn't be loading it as a module. E.g.
JSON loaded via some future import mechanism would never be GC'd. In the
fictional example of the huge MMORPG
with many planets, you'd likely want to load the things you care about
GC'ing via fetch() or some other mechanism
that allows managing lifetime yourself.[1]

Cheers,
Jan

On Wed, Jul 1, 2020 at 5:00 AM  wrote:

> Send es-discuss mailing list submissions to
> es-discuss@mozilla.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://mail.mozilla.org/listinfo/es-discuss
> or, via email, send a message with subject or body 'help' to
> es-discuss-requ...@mozilla.org
>
> You can reach the person managing the list at
> es-discuss-ow...@mozilla.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of es-discuss digest..."
> Today's Topics:
>
>1. Re: Are ES Modules garbage collected? If so, do they
>   re-execute on next import? (Isiah Meadows)
>
>
>
> -- Forwarded message --
> From: Isiah Meadows 
> To: Andrea Giammarchi 
> Cc: Isiah Meadows , Gus Caplan ,
> es-discuss 
> Bcc:
> Date: Wed, 1 Jul 2020 00:47:30 -0700
> Subject: Re: Are ES Modules garbage collected? If so, do they re-execute
> on next import?
> That's part of the caching I'm referring to. And if the cache entry
> for it has been evicted, I would *not* expect it to necessarily return
> the same instance, consistent with the behavior with `require` and
> `require.cache` in Node (and similar with most other module loaders
> that support cache eviction).
>
> -
>
> Isiah Meadows
> cont...@isiahmeadows.com
> www.isiahmeadows.com
>
> On Tue, Jun 30, 2020 at 11:57 PM Andrea Giammarchi
>  wrote:
> >
> > even if dereferenced, a dynamic import could re-reference it any time,
> and I would expect it to still be the same module, it'd be a surprise
> otherwise (cached things, same namespace checks, etc).
> >
> > On Wed, Jul 1, 2020 at 7:33 AM Isiah Meadows 
> wrote:
> >>
> >> Just to expand on that, if the module record itself is dereferenced
> >> (like if it's evicted from the cache somehow), then yes, it should be
> >> collected as appropriate. However, I'm not aware of any major
> >> implementation that offers that functionality.
> >>
> >> -
> >>
> >> Isiah Meadows
> >> cont...@isiahmeadows.com
> >> www.isiahmeadows.com
> >>
> >> On Tue, Jun 30, 2020 at 6:22 PM Gus Caplan  wrote:
> >> >
> >> > Modules in the spec are cached by specifier by modules that import
> them. Modules in major implementations are additionally cached for the
> entire realm by absolute URLs. I would say that for actual code (functions
> and classes and whatnot) leaks aren't really a problem. Even if you import
> a ton of levels, that's not that much memory. The main concern I've seen
> raised is JSON modules, where once you import them the JSON object, which
> can be quite large, will never be collected. Of course, there is a simple
> solution to that (fetch) so it isn't a world ending problem.
> >> >
> >> > On Tue, Jun 30, 2020 at 7:41 PM #!/JoePea  wrote:
> >> >>
> >> >> I am curious: can modules be garbage collected if the exports are not
> >> >> references by anything anymore? And if so, will the module be
> >> >> re-evaluated the next time it is imported?
> >> >>
> >> >> I haven't tried an experiment to answer this yet. I'll be back to
> post
> >> >> findings if someone doesn't post an official answer first.
> >> >>
> >> >> I'm thinking about code longevity. For example, if we make
> >> >> long-running web-based applications with many routes and features
> (for
> >> >> sake of example imagine a desktop environment, or a MMORPG game, with
> >> >> apps or components that are loaded within the same context). Over
> >> >> time, if imports are not collected, then it means we have a memory
> >> >> leak.
> >> >>
> >> >> Imagine, for example, an infinite-universe MMORPG where you can land
> >> >> on different planets where the code for features of a planet are
> >> >> provided by third parties as 

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Mark S. Miller]

No, definitely not. The table from specifiers to module instances is
indexed by specifiers. Specifiers are strings, so this table is not weak.
It is not a "cache" in the sense that it is allowed to drop things. Rather
it is a registry of module instances. Only a registry as a whole can be
gc'ed, and which point that context is no longer around for instantiating
or reinstantiating modules.

As you suggest, if it could drop things because of GC that it would then
need to regenerate, that would expose the non-determinism of gc. That would
be a big deal. We carefully designed WeakMaps so that gc was
non-observable. WeakMaps introduce no observable non-determinism. WeakRefs
alone expose the non-determinism of gc, and are kept well quarantined from
the rest of the language.


On Tue, Jun 30, 2020 at 5:42 PM #!/JoePea  wrote:

> I am curious: can modules be garbage collected if the exports are not
> references by anything anymore? And if so, will the module be
> re-evaluated the next time it is imported?
>
> I haven't tried an experiment to answer this yet. I'll be back to post
> findings if someone doesn't post an official answer first.
>
> I'm thinking about code longevity. For example, if we make
> long-running web-based applications with many routes and features (for
> sake of example imagine a desktop environment, or a MMORPG game, with
> apps or components that are loaded within the same context). Over
> time, if imports are not collected, then it means we have a memory
> leak.
>
> Imagine, for example, an infinite-universe MMORPG where you can land
> on different planets where the code for features of a planet are
> provided by third parties as ES Modules. I know, this might not be a
> safe idea to import any code into an app, but just imagine it for sake
> of example (imagine we have a continuous integration system to test
> and verify code security, or something, before that code is allowed to
> be consumed in the app). Imagine you play this app for many many days,
> and visit many places, and you leave the app running the whole time
> (because farming for resources is disabled if the app is not running,
> or something).
>
> I would imagine that we want unused modules (when we leave a planet,
> for example) to be (destroyed) garbage collected so that we don't
> waste memory.
>
> #!/JoePea
> ___
> es-discuss mailing list
> es-discuss@mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss
>


-- 
  Cheers,
  --MarkM
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Isiah Meadows]

That's part of the caching I'm referring to. And if the cache entry
for it has been evicted, I would *not* expect it to necessarily return
the same instance, consistent with the behavior with `require` and
`require.cache` in Node (and similar with most other module loaders
that support cache eviction).

-

Isiah Meadows
cont...@isiahmeadows.com
www.isiahmeadows.com

On Tue, Jun 30, 2020 at 11:57 PM Andrea Giammarchi
 wrote:
>
> even if dereferenced, a dynamic import could re-reference it any time, and I 
> would expect it to still be the same module, it'd be a surprise otherwise 
> (cached things, same namespace checks, etc).
>
> On Wed, Jul 1, 2020 at 7:33 AM Isiah Meadows  wrote:
>>
>> Just to expand on that, if the module record itself is dereferenced
>> (like if it's evicted from the cache somehow), then yes, it should be
>> collected as appropriate. However, I'm not aware of any major
>> implementation that offers that functionality.
>>
>> -
>>
>> Isiah Meadows
>> cont...@isiahmeadows.com
>> www.isiahmeadows.com
>>
>> On Tue, Jun 30, 2020 at 6:22 PM Gus Caplan  wrote:
>> >
>> > Modules in the spec are cached by specifier by modules that import them. 
>> > Modules in major implementations are additionally cached for the entire 
>> > realm by absolute URLs. I would say that for actual code (functions and 
>> > classes and whatnot) leaks aren't really a problem. Even if you import a 
>> > ton of levels, that's not that much memory. The main concern I've seen 
>> > raised is JSON modules, where once you import them the JSON object, which 
>> > can be quite large, will never be collected. Of course, there is a simple 
>> > solution to that (fetch) so it isn't a world ending problem.
>> >
>> > On Tue, Jun 30, 2020 at 7:41 PM #!/JoePea  wrote:
>> >>
>> >> I am curious: can modules be garbage collected if the exports are not
>> >> references by anything anymore? And if so, will the module be
>> >> re-evaluated the next time it is imported?
>> >>
>> >> I haven't tried an experiment to answer this yet. I'll be back to post
>> >> findings if someone doesn't post an official answer first.
>> >>
>> >> I'm thinking about code longevity. For example, if we make
>> >> long-running web-based applications with many routes and features (for
>> >> sake of example imagine a desktop environment, or a MMORPG game, with
>> >> apps or components that are loaded within the same context). Over
>> >> time, if imports are not collected, then it means we have a memory
>> >> leak.
>> >>
>> >> Imagine, for example, an infinite-universe MMORPG where you can land
>> >> on different planets where the code for features of a planet are
>> >> provided by third parties as ES Modules. I know, this might not be a
>> >> safe idea to import any code into an app, but just imagine it for sake
>> >> of example (imagine we have a continuous integration system to test
>> >> and verify code security, or something, before that code is allowed to
>> >> be consumed in the app). Imagine you play this app for many many days,
>> >> and visit many places, and you leave the app running the whole time
>> >> (because farming for resources is disabled if the app is not running,
>> >> or something).
>> >>
>> >> I would imagine that we want unused modules (when we leave a planet,
>> >> for example) to be (destroyed) garbage collected so that we don't
>> >> waste memory.
>> >>
>> >> #!/JoePea
>> >> ___
>> >> es-discuss mailing list
>> >> es-discuss@mozilla.org
>> >> https://mail.mozilla.org/listinfo/es-discuss
>> >
>> > ___
>> > es-discuss mailing list
>> > es-discuss@mozilla.org
>> > https://mail.mozilla.org/listinfo/es-discuss
>> ___
>> es-discuss mailing list
>> es-discuss@mozilla.org
>> https://mail.mozilla.org/listinfo/es-discuss
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Andrea Giammarchi]

even if dereferenced, a dynamic import could re-reference it any time, and
I would expect it to still be the same module, it'd be a surprise otherwise
(cached things, same namespace checks, etc).

On Wed, Jul 1, 2020 at 7:33 AM Isiah Meadows 
wrote:

> Just to expand on that, if the module record itself is dereferenced
> (like if it's evicted from the cache somehow), then yes, it should be
> collected as appropriate. However, I'm not aware of any major
> implementation that offers that functionality.
>
> -
>
> Isiah Meadows
> cont...@isiahmeadows.com
> www.isiahmeadows.com
>
> On Tue, Jun 30, 2020 at 6:22 PM Gus Caplan  wrote:
> >
> > Modules in the spec are cached by specifier by modules that import them.
> Modules in major implementations are additionally cached for the entire
> realm by absolute URLs. I would say that for actual code (functions and
> classes and whatnot) leaks aren't really a problem. Even if you import a
> ton of levels, that's not that much memory. The main concern I've seen
> raised is JSON modules, where once you import them the JSON object, which
> can be quite large, will never be collected. Of course, there is a simple
> solution to that (fetch) so it isn't a world ending problem.
> >
> > On Tue, Jun 30, 2020 at 7:41 PM #!/JoePea  wrote:
> >>
> >> I am curious: can modules be garbage collected if the exports are not
> >> references by anything anymore? And if so, will the module be
> >> re-evaluated the next time it is imported?
> >>
> >> I haven't tried an experiment to answer this yet. I'll be back to post
> >> findings if someone doesn't post an official answer first.
> >>
> >> I'm thinking about code longevity. For example, if we make
> >> long-running web-based applications with many routes and features (for
> >> sake of example imagine a desktop environment, or a MMORPG game, with
> >> apps or components that are loaded within the same context). Over
> >> time, if imports are not collected, then it means we have a memory
> >> leak.
> >>
> >> Imagine, for example, an infinite-universe MMORPG where you can land
> >> on different planets where the code for features of a planet are
> >> provided by third parties as ES Modules. I know, this might not be a
> >> safe idea to import any code into an app, but just imagine it for sake
> >> of example (imagine we have a continuous integration system to test
> >> and verify code security, or something, before that code is allowed to
> >> be consumed in the app). Imagine you play this app for many many days,
> >> and visit many places, and you leave the app running the whole time
> >> (because farming for resources is disabled if the app is not running,
> >> or something).
> >>
> >> I would imagine that we want unused modules (when we leave a planet,
> >> for example) to be (destroyed) garbage collected so that we don't
> >> waste memory.
> >>
> >> #!/JoePea
> >> ___
> >> es-discuss mailing list
> >> es-discuss@mozilla.org
> >> https://mail.mozilla.org/listinfo/es-discuss
> >
> > ___
> > es-discuss mailing list
> > es-discuss@mozilla.org
> > https://mail.mozilla.org/listinfo/es-discuss
> ___
> es-discuss mailing list
> es-discuss@mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss
>
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss