Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Guy Bedford]

The spec and implementation are exactly defined like this - v8 directly
provides HostResolveImportedModule exactly to the spec, returning a module
object for a given specifier and module parent. By default in v8 modules
are GC'able objects... they don't implement registries currently. There's
just some work needed there to work out why, for example, in Node.js they
aren't possible to GC.

On Sat, 18 Jul 2020 at 15:42, Isiah Meadows 
wrote:

> For the web, one could lock it down with CORS-like restrictions, too,
> so a script from one domain isn't allowed to modify cached entries
> from another domain. It's not insurmountable.
>
> This is part of why I feel the invariant should be removed and the
> spec should just internally resolve static imports to dedupe them and
> then resolve them accordingly when executing them, rather than
> consulting external caches every time. (It's also easier for the
> engine to handle, assuming they don't do this already.)
>
> -
>
> Isiah Meadows
> cont...@isiahmeadows.com
> www.isiahmeadows.com
>
> On Thu, Jul 16, 2020 at 6:44 AM Andrea Giammarchi
>  wrote:
> >
> > FWIW explicit eviction is not only fine, if you own the code that does
> that, but the only way I can code cover 100% all the branches of my
> libraries. The issue here is the untrusted Web, where I'd never expect any
> 3rd parts library to evict a module I am using within my code ... like ...
> ever.
> >
> > Accordingly, I think Allen, if it was Allen, made the right call for ESM.
> >
> > On Thu, Jul 16, 2020 at 3:23 PM Guy Bedford 
> wrote:
> >>
> >> Node.js in the CommonJS loader and dynamic loaders like SystemJS have
> supported module unloading for many years by permitting eviction from the
> loader registry. Evicting the full parent tree was the traditional
> reloading workflow in SystemJS, but live binding pushes are also permitted
> in SystemJS as well for this.
> >>
> >> I agree there are issues and invariants of course to consider from a
> theoretical perspective, but those are decisions to be made in terms of
> what invariants are valued, and I don't feel they are necessarily absolute
> constraints. These decisions should be made based on what is best for the
> JS users and engines. Not that I feel strongly this should be a requirement
> but that it should still be open to consideration.
> >>
> >> I'm not sure it was Allen's intention to ban any concept of reloading
> modules when defining the idempotency requirement for the host resolve
> function. Perhaps he could speak to that if he's around.
> >>
> >>
> >> On Tue, 14 Jul 2020 at 23:05, Mark S. Miller  wrote:
> >>>
> >>> Only a module registry as a whole may be GCed. During the lifetime of
> any one module registry, it can only grow. No other solution is possible.
> >>>
> >>> Btw, I remember being surprised ages ago when the same issue came up
> for the Java ClassLoader. A classLoader holds on to all the classes it ever
> loaded. Each class holds onto its classLoader. Each instance holds on to
> its class. During the lifetime of a classLoader or any of its classes, the
> graph of that classLoader and its classes can only grow new classes. Not
> until the classLoader and all of its classes are unreachable at the same
> time can any of them be collected. This was equally unfortunate,
> surprising, and inescapable.
> >>>
> >>>
> >>>
> >>> On Tue, Jul 14, 2020 at 10:16 PM #!/JoePea  wrote:
> 
>  How can we ensure that long-running applications (even if
> theoretical),
>  that may load and unload an unlimited number of new modules over time
>  (f.e. routes in a web page specified by 3rd parties as time
>  progresses), not leak memory?
> 
>  Even if it is theoretical, I don't like the thought of something that
>  only ever allocates memory that will never be freed.
> 
>  Is someone working on a solution for this?
> 
> 
>  #!/JoePea
> 
>  On Wed, Jul 1, 2020 at 6:16 AM Mark S. Miller 
> wrote:
>  >
>  > No, definitely not. The table from specifiers to module instances
> is indexed by specifiers. Specifiers are strings, so this table is not
> weak. It is not a "cache" in the sense that it is allowed to drop things.
> Rather it is a registry of module instances. Only a registry as a whole can
> be gc'ed, and which point that context is no longer around for
> instantiating or reinstantiating modules.
>  >
>  > As you suggest, if it could drop things because of GC that it would
> then need to regenerate, that would expose the non-determinism of gc. That
> would be a big deal. We carefully designed WeakMaps so that gc was
> non-observable. WeakMaps introduce no observable non-determinism. WeakRefs
> alone expose the non-determinism of gc, and are kept well quarantined from
> the rest of the language.
>  >
>  >
>  > On Tue, Jun 30, 2020 at 5:42 PM #!/JoePea  wrote:
>  >>
>  >> I am curious: can modules be garbage collected if the exports are
> not
>  >> referenc

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Isiah Meadows]

For the web, one could lock it down with CORS-like restrictions, too,
so a script from one domain isn't allowed to modify cached entries
from another domain. It's not insurmountable.

This is part of why I feel the invariant should be removed and the
spec should just internally resolve static imports to dedupe them and
then resolve them accordingly when executing them, rather than
consulting external caches every time. (It's also easier for the
engine to handle, assuming they don't do this already.)

-

Isiah Meadows
cont...@isiahmeadows.com
www.isiahmeadows.com

On Thu, Jul 16, 2020 at 6:44 AM Andrea Giammarchi
 wrote:
>
> FWIW explicit eviction is not only fine, if you own the code that does that, 
> but the only way I can code cover 100% all the branches of my libraries. The 
> issue here is the untrusted Web, where I'd never expect any 3rd parts library 
> to evict a module I am using within my code ... like ... ever.
>
> Accordingly, I think Allen, if it was Allen, made the right call for ESM.
>
> On Thu, Jul 16, 2020 at 3:23 PM Guy Bedford  wrote:
>>
>> Node.js in the CommonJS loader and dynamic loaders like SystemJS have 
>> supported module unloading for many years by permitting eviction from the 
>> loader registry. Evicting the full parent tree was the traditional reloading 
>> workflow in SystemJS, but live binding pushes are also permitted in SystemJS 
>> as well for this.
>>
>> I agree there are issues and invariants of course to consider from a 
>> theoretical perspective, but those are decisions to be made in terms of what 
>> invariants are valued, and I don't feel they are necessarily absolute 
>> constraints. These decisions should be made based on what is best for the JS 
>> users and engines. Not that I feel strongly this should be a requirement but 
>> that it should still be open to consideration.
>>
>> I'm not sure it was Allen's intention to ban any concept of reloading 
>> modules when defining the idempotency requirement for the host resolve 
>> function. Perhaps he could speak to that if he's around.
>>
>>
>> On Tue, 14 Jul 2020 at 23:05, Mark S. Miller  wrote:
>>>
>>> Only a module registry as a whole may be GCed. During the lifetime of any 
>>> one module registry, it can only grow. No other solution is possible.
>>>
>>> Btw, I remember being surprised ages ago when the same issue came up for 
>>> the Java ClassLoader. A classLoader holds on to all the classes it ever 
>>> loaded. Each class holds onto its classLoader. Each instance holds on to 
>>> its class. During the lifetime of a classLoader or any of its classes, the 
>>> graph of that classLoader and its classes can only grow new classes. Not 
>>> until the classLoader and all of its classes are unreachable at the same 
>>> time can any of them be collected. This was equally unfortunate, 
>>> surprising, and inescapable.
>>>
>>>
>>>
>>> On Tue, Jul 14, 2020 at 10:16 PM #!/JoePea  wrote:

 How can we ensure that long-running applications (even if theoretical),
 that may load and unload an unlimited number of new modules over time
 (f.e. routes in a web page specified by 3rd parties as time
 progresses), not leak memory?

 Even if it is theoretical, I don't like the thought of something that
 only ever allocates memory that will never be freed.

 Is someone working on a solution for this?


 #!/JoePea

 On Wed, Jul 1, 2020 at 6:16 AM Mark S. Miller  wrote:
 >
 > No, definitely not. The table from specifiers to module instances is 
 > indexed by specifiers. Specifiers are strings, so this table is not 
 > weak. It is not a "cache" in the sense that it is allowed to drop 
 > things. Rather it is a registry of module instances. Only a registry as 
 > a whole can be gc'ed, and which point that context is no longer around 
 > for instantiating or reinstantiating modules.
 >
 > As you suggest, if it could drop things because of GC that it would then 
 > need to regenerate, that would expose the non-determinism of gc. That 
 > would be a big deal. We carefully designed WeakMaps so that gc was 
 > non-observable. WeakMaps introduce no observable non-determinism. 
 > WeakRefs alone expose the non-determinism of gc, and are kept well 
 > quarantined from the rest of the language.
 >
 >
 > On Tue, Jun 30, 2020 at 5:42 PM #!/JoePea  wrote:
 >>
 >> I am curious: can modules be garbage collected if the exports are not
 >> references by anything anymore? And if so, will the module be
 >> re-evaluated the next time it is imported?
 >>
 >> I haven't tried an experiment to answer this yet. I'll be back to post
 >> findings if someone doesn't post an official answer first.
 >>
 >> I'm thinking about code longevity. For example, if we make
 >> long-running web-based applications with many routes and features (for
 >> sake of example imagine a desktop environment, or a M

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Herby Vojčík]

On 16. 7. 2020 15:23, Guy Bedford wrote:
Node.js in the CommonJS loader and dynamic loaders like SystemJS have 
supported module unloading for many years by permitting eviction from 
the loader registry. Evicting the full parent tree was the traditional 
reloading workflow in SystemJS, but live binding pushes are also 
permitted in SystemJS as well for this.


I agree there are issues and invariants of course to consider from a 
theoretical perspective, but those are decisions to be made in terms of 
what invariants are valued, and I don't feel they are necessarily 


I understand there may be problems with static imports that are 
statically analyzed, and there "only GC module loader as whole" makes 
sense, but maybe, it would be possible to treat _dynamically loaded_ 
modules different way? Just thinking aloud.


absolute constraints. These decisions should be made based on what is 
best for the JS users and engines. Not that I feel strongly this should 
be a requirement but that it should still be open to consideration.


I'm not sure it was Allen's intention to ban any concept of reloading 
modules when defining the idempotency requirement for the host resolve 
function. Perhaps he could speak to that if he's around.



On Tue, 14 Jul 2020 at 23:05, Mark S. Miller > wrote:


Only a module registry as a whole may be GCed. During the lifetime
of any one module registry, it can only grow. No other solution is
possible.

Btw, I remember being surprised ages ago when the same issue came up
for the Java ClassLoader. A classLoader holds on to all the classes
it ever loaded. Each class holds onto its classLoader. Each instance
holds on to its class. During the lifetime of a classLoader or any
of its classes, the graph of that classLoader and its classes can
only grow new classes. Not until the classLoader and all of its
classes are unreachable at the same time can any of them be
collected. This was equally unfortunate, surprising, and inescapable.



On Tue, Jul 14, 2020 at 10:16 PM #!/JoePea mailto:j...@trusktr.io>> wrote:

How can we ensure that long-running applications (even if
theoretical),
that may load and unload an unlimited number of new modules over
time
(f.e. routes in a web page specified by 3rd parties as time
progresses), not leak memory?

Even if it is theoretical, I don't like the thought of something
that
only ever allocates memory that will never be freed.

Is someone working on a solution for this?


#!/JoePea

On Wed, Jul 1, 2020 at 6:16 AM Mark S. Miller mailto:erig...@gmail.com>> wrote:
 >
 > No, definitely not. The table from specifiers to module
instances is indexed by specifiers. Specifiers are strings, so
this table is not weak. It is not a "cache" in the sense that it
is allowed to drop things. Rather it is a registry of module
instances. Only a registry as a whole can be gc'ed, and which
point that context is no longer around for instantiating or
reinstantiating modules.
 >
 > As you suggest, if it could drop things because of GC that it
would then need to regenerate, that would expose the
non-determinism of gc. That would be a big deal. We carefully
designed WeakMaps so that gc was non-observable. WeakMaps
introduce no observable non-determinism. WeakRefs alone expose
the non-determinism of gc, and are kept well quarantined from
the rest of the language.
 >
 >
 > On Tue, Jun 30, 2020 at 5:42 PM #!/JoePea mailto:j...@trusktr.io>> wrote:
 >>
 >> I am curious: can modules be garbage collected if the
exports are not
 >> references by anything anymore? And if so, will the module be
 >> re-evaluated the next time it is imported?
 >>
 >> I haven't tried an experiment to answer this yet. I'll be
back to post
 >> findings if someone doesn't post an official answer first.
 >>
 >> I'm thinking about code longevity. For example, if we make
 >> long-running web-based applications with many routes and
features (for
 >> sake of example imagine a desktop environment, or a MMORPG
game, with
 >> apps or components that are loaded within the same context).
Over
 >> time, if imports are not collected, then it means we have a
memory
 >> leak.
 >>
 >> Imagine, for example, an infinite-universe MMORPG where you
can land
 >> on different planets where the code for features of a planet are
 >> provided by third parties as ES Modules. I know, this might
not be a
 >> safe idea to import any code into an app, but just imagine
it for sake
 >> of example (imagine we 

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Andrea Giammarchi]

On a second thought ... couldn't `import.meta.cache`, or something similar,
be implemented in NodeJS only?

On Thu, Jul 16, 2020 at 3:44 PM Andrea Giammarchi <
andrea.giammar...@gmail.com> wrote:

> FWIW explicit eviction is not only fine, if you own the code that does
> that, but the only way I can code cover 100% all the branches of my
> libraries. The issue here is the untrusted Web, where I'd never expect any
> 3rd parts library to evict a module I am using within my code ... like ...
> ever.
>
> Accordingly, I think Allen, if it was Allen, made the right call for ESM.
>
> On Thu, Jul 16, 2020 at 3:23 PM Guy Bedford  wrote:
>
>> Node.js in the CommonJS loader and dynamic loaders like SystemJS have
>> supported module unloading for many years by permitting eviction from the
>> loader registry. Evicting the full parent tree was the traditional
>> reloading workflow in SystemJS, but live binding pushes are also permitted
>> in SystemJS as well for this.
>>
>> I agree there are issues and invariants of course to consider from a
>> theoretical perspective, but those are decisions to be made in terms of
>> what invariants are valued, and I don't feel they are necessarily absolute
>> constraints. These decisions should be made based on what is best for
>> the JS users and engines. Not that I feel strongly this should be a
>> requirement but that it should still be open to consideration.
>>
>> I'm not sure it was Allen's intention to ban any concept of reloading
>> modules when defining the idempotency requirement for the host resolve
>> function. Perhaps he could speak to that if he's around.
>>
>>
>> On Tue, 14 Jul 2020 at 23:05, Mark S. Miller  wrote:
>>
>>> Only a module registry as a whole may be GCed. During the lifetime of
>>> any one module registry, it can only grow. No other solution is possible.
>>>
>>> Btw, I remember being surprised ages ago when the same issue came up for
>>> the Java ClassLoader. A classLoader holds on to all the classes it ever
>>> loaded. Each class holds onto its classLoader. Each instance holds on to
>>> its class. During the lifetime of a classLoader or any of its classes, the
>>> graph of that classLoader and its classes can only grow new classes. Not
>>> until the classLoader and all of its classes are unreachable at the same
>>> time can any of them be collected. This was equally unfortunate,
>>> surprising, and inescapable.
>>>
>>>
>>>
>>> On Tue, Jul 14, 2020 at 10:16 PM #!/JoePea  wrote:
>>>
 How can we ensure that long-running applications (even if theoretical),
 that may load and unload an unlimited number of new modules over time
 (f.e. routes in a web page specified by 3rd parties as time
 progresses), not leak memory?

 Even if it is theoretical, I don't like the thought of something that
 only ever allocates memory that will never be freed.

 Is someone working on a solution for this?


 #!/JoePea

 On Wed, Jul 1, 2020 at 6:16 AM Mark S. Miller 
 wrote:
 >
 > No, definitely not. The table from specifiers to module instances is
 indexed by specifiers. Specifiers are strings, so this table is not weak.
 It is not a "cache" in the sense that it is allowed to drop things. Rather
 it is a registry of module instances. Only a registry as a whole can be
 gc'ed, and which point that context is no longer around for instantiating
 or reinstantiating modules.
 >
 > As you suggest, if it could drop things because of GC that it would
 then need to regenerate, that would expose the non-determinism of gc. That
 would be a big deal. We carefully designed WeakMaps so that gc was
 non-observable. WeakMaps introduce no observable non-determinism. WeakRefs
 alone expose the non-determinism of gc, and are kept well quarantined from
 the rest of the language.
 >
 >
 > On Tue, Jun 30, 2020 at 5:42 PM #!/JoePea  wrote:
 >>
 >> I am curious: can modules be garbage collected if the exports are not
 >> references by anything anymore? And if so, will the module be
 >> re-evaluated the next time it is imported?
 >>
 >> I haven't tried an experiment to answer this yet. I'll be back to
 post
 >> findings if someone doesn't post an official answer first.
 >>
 >> I'm thinking about code longevity. For example, if we make
 >> long-running web-based applications with many routes and features
 (for
 >> sake of example imagine a desktop environment, or a MMORPG game, with
 >> apps or components that are loaded within the same context). Over
 >> time, if imports are not collected, then it means we have a memory
 >> leak.
 >>
 >> Imagine, for example, an infinite-universe MMORPG where you can land
 >> on different planets where the code for features of a planet are
 >> provided by third parties as ES Modules. I know, this might not be a
 >> safe idea to import any code into an app, but just i

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Andrea Giammarchi]

FWIW explicit eviction is not only fine, if you own the code that does
that, but the only way I can code cover 100% all the branches of my
libraries. The issue here is the untrusted Web, where I'd never expect any
3rd parts library to evict a module I am using within my code ... like ...
ever.

Accordingly, I think Allen, if it was Allen, made the right call for ESM.

On Thu, Jul 16, 2020 at 3:23 PM Guy Bedford  wrote:

> Node.js in the CommonJS loader and dynamic loaders like SystemJS have
> supported module unloading for many years by permitting eviction from the
> loader registry. Evicting the full parent tree was the traditional
> reloading workflow in SystemJS, but live binding pushes are also permitted
> in SystemJS as well for this.
>
> I agree there are issues and invariants of course to consider from a
> theoretical perspective, but those are decisions to be made in terms of
> what invariants are valued, and I don't feel they are necessarily absolute
> constraints. These decisions should be made based on what is best for
> the JS users and engines. Not that I feel strongly this should be a
> requirement but that it should still be open to consideration.
>
> I'm not sure it was Allen's intention to ban any concept of reloading
> modules when defining the idempotency requirement for the host resolve
> function. Perhaps he could speak to that if he's around.
>
>
> On Tue, 14 Jul 2020 at 23:05, Mark S. Miller  wrote:
>
>> Only a module registry as a whole may be GCed. During the lifetime of any
>> one module registry, it can only grow. No other solution is possible.
>>
>> Btw, I remember being surprised ages ago when the same issue came up for
>> the Java ClassLoader. A classLoader holds on to all the classes it ever
>> loaded. Each class holds onto its classLoader. Each instance holds on to
>> its class. During the lifetime of a classLoader or any of its classes, the
>> graph of that classLoader and its classes can only grow new classes. Not
>> until the classLoader and all of its classes are unreachable at the same
>> time can any of them be collected. This was equally unfortunate,
>> surprising, and inescapable.
>>
>>
>>
>> On Tue, Jul 14, 2020 at 10:16 PM #!/JoePea  wrote:
>>
>>> How can we ensure that long-running applications (even if theoretical),
>>> that may load and unload an unlimited number of new modules over time
>>> (f.e. routes in a web page specified by 3rd parties as time
>>> progresses), not leak memory?
>>>
>>> Even if it is theoretical, I don't like the thought of something that
>>> only ever allocates memory that will never be freed.
>>>
>>> Is someone working on a solution for this?
>>>
>>>
>>> #!/JoePea
>>>
>>> On Wed, Jul 1, 2020 at 6:16 AM Mark S. Miller  wrote:
>>> >
>>> > No, definitely not. The table from specifiers to module instances is
>>> indexed by specifiers. Specifiers are strings, so this table is not weak.
>>> It is not a "cache" in the sense that it is allowed to drop things. Rather
>>> it is a registry of module instances. Only a registry as a whole can be
>>> gc'ed, and which point that context is no longer around for instantiating
>>> or reinstantiating modules.
>>> >
>>> > As you suggest, if it could drop things because of GC that it would
>>> then need to regenerate, that would expose the non-determinism of gc. That
>>> would be a big deal. We carefully designed WeakMaps so that gc was
>>> non-observable. WeakMaps introduce no observable non-determinism. WeakRefs
>>> alone expose the non-determinism of gc, and are kept well quarantined from
>>> the rest of the language.
>>> >
>>> >
>>> > On Tue, Jun 30, 2020 at 5:42 PM #!/JoePea  wrote:
>>> >>
>>> >> I am curious: can modules be garbage collected if the exports are not
>>> >> references by anything anymore? And if so, will the module be
>>> >> re-evaluated the next time it is imported?
>>> >>
>>> >> I haven't tried an experiment to answer this yet. I'll be back to post
>>> >> findings if someone doesn't post an official answer first.
>>> >>
>>> >> I'm thinking about code longevity. For example, if we make
>>> >> long-running web-based applications with many routes and features (for
>>> >> sake of example imagine a desktop environment, or a MMORPG game, with
>>> >> apps or components that are loaded within the same context). Over
>>> >> time, if imports are not collected, then it means we have a memory
>>> >> leak.
>>> >>
>>> >> Imagine, for example, an infinite-universe MMORPG where you can land
>>> >> on different planets where the code for features of a planet are
>>> >> provided by third parties as ES Modules. I know, this might not be a
>>> >> safe idea to import any code into an app, but just imagine it for sake
>>> >> of example (imagine we have a continuous integration system to test
>>> >> and verify code security, or something, before that code is allowed to
>>> >> be consumed in the app). Imagine you play this app for many many days,
>>> >> and visit many places, and you leave the app running the w

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Guy Bedford]

Node.js in the CommonJS loader and dynamic loaders like SystemJS have
supported module unloading for many years by permitting eviction from the
loader registry. Evicting the full parent tree was the traditional
reloading workflow in SystemJS, but live binding pushes are also permitted
in SystemJS as well for this.

I agree there are issues and invariants of course to consider from a
theoretical perspective, but those are decisions to be made in terms of
what invariants are valued, and I don't feel they are necessarily absolute
constraints. These decisions should be made based on what is best for
the JS users and engines. Not that I feel strongly this should be a
requirement but that it should still be open to consideration.

I'm not sure it was Allen's intention to ban any concept of reloading
modules when defining the idempotency requirement for the host resolve
function. Perhaps he could speak to that if he's around.


On Tue, 14 Jul 2020 at 23:05, Mark S. Miller  wrote:

> Only a module registry as a whole may be GCed. During the lifetime of any
> one module registry, it can only grow. No other solution is possible.
>
> Btw, I remember being surprised ages ago when the same issue came up for
> the Java ClassLoader. A classLoader holds on to all the classes it ever
> loaded. Each class holds onto its classLoader. Each instance holds on to
> its class. During the lifetime of a classLoader or any of its classes, the
> graph of that classLoader and its classes can only grow new classes. Not
> until the classLoader and all of its classes are unreachable at the same
> time can any of them be collected. This was equally unfortunate,
> surprising, and inescapable.
>
>
>
> On Tue, Jul 14, 2020 at 10:16 PM #!/JoePea  wrote:
>
>> How can we ensure that long-running applications (even if theoretical),
>> that may load and unload an unlimited number of new modules over time
>> (f.e. routes in a web page specified by 3rd parties as time
>> progresses), not leak memory?
>>
>> Even if it is theoretical, I don't like the thought of something that
>> only ever allocates memory that will never be freed.
>>
>> Is someone working on a solution for this?
>>
>>
>> #!/JoePea
>>
>> On Wed, Jul 1, 2020 at 6:16 AM Mark S. Miller  wrote:
>> >
>> > No, definitely not. The table from specifiers to module instances is
>> indexed by specifiers. Specifiers are strings, so this table is not weak.
>> It is not a "cache" in the sense that it is allowed to drop things. Rather
>> it is a registry of module instances. Only a registry as a whole can be
>> gc'ed, and which point that context is no longer around for instantiating
>> or reinstantiating modules.
>> >
>> > As you suggest, if it could drop things because of GC that it would
>> then need to regenerate, that would expose the non-determinism of gc. That
>> would be a big deal. We carefully designed WeakMaps so that gc was
>> non-observable. WeakMaps introduce no observable non-determinism. WeakRefs
>> alone expose the non-determinism of gc, and are kept well quarantined from
>> the rest of the language.
>> >
>> >
>> > On Tue, Jun 30, 2020 at 5:42 PM #!/JoePea  wrote:
>> >>
>> >> I am curious: can modules be garbage collected if the exports are not
>> >> references by anything anymore? And if so, will the module be
>> >> re-evaluated the next time it is imported?
>> >>
>> >> I haven't tried an experiment to answer this yet. I'll be back to post
>> >> findings if someone doesn't post an official answer first.
>> >>
>> >> I'm thinking about code longevity. For example, if we make
>> >> long-running web-based applications with many routes and features (for
>> >> sake of example imagine a desktop environment, or a MMORPG game, with
>> >> apps or components that are loaded within the same context). Over
>> >> time, if imports are not collected, then it means we have a memory
>> >> leak.
>> >>
>> >> Imagine, for example, an infinite-universe MMORPG where you can land
>> >> on different planets where the code for features of a planet are
>> >> provided by third parties as ES Modules. I know, this might not be a
>> >> safe idea to import any code into an app, but just imagine it for sake
>> >> of example (imagine we have a continuous integration system to test
>> >> and verify code security, or something, before that code is allowed to
>> >> be consumed in the app). Imagine you play this app for many many days,
>> >> and visit many places, and you leave the app running the whole time
>> >> (because farming for resources is disabled if the app is not running,
>> >> or something).
>> >>
>> >> I would imagine that we want unused modules (when we leave a planet,
>> >> for example) to be (destroyed) garbage collected so that we don't
>> >> waste memory.
>> >>
>> >> #!/JoePea
>> >> ___
>> >> es-discuss mailing list
>> >> es-discuss@mozilla.org
>> >> https://mail.mozilla.org/listinfo/es-discuss
>> >
>> >
>> >
>> > --
>> >   Cheers,
>> >   --MarkM
>>
>
>
> -

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Mark S. Miller]

Only a module registry as a whole may be GCed. During the lifetime of any
one module registry, it can only grow. No other solution is possible.

Btw, I remember being surprised ages ago when the same issue came up for
the Java ClassLoader. A classLoader holds on to all the classes it ever
loaded. Each class holds onto its classLoader. Each instance holds on to
its class. During the lifetime of a classLoader or any of its classes, the
graph of that classLoader and its classes can only grow new classes. Not
until the classLoader and all of its classes are unreachable at the same
time can any of them be collected. This was equally unfortunate,
surprising, and inescapable.



On Tue, Jul 14, 2020 at 10:16 PM #!/JoePea  wrote:

> How can we ensure that long-running applications (even if theoretical),
> that may load and unload an unlimited number of new modules over time
> (f.e. routes in a web page specified by 3rd parties as time
> progresses), not leak memory?
>
> Even if it is theoretical, I don't like the thought of something that
> only ever allocates memory that will never be freed.
>
> Is someone working on a solution for this?
>
>
> #!/JoePea
>
> On Wed, Jul 1, 2020 at 6:16 AM Mark S. Miller  wrote:
> >
> > No, definitely not. The table from specifiers to module instances is
> indexed by specifiers. Specifiers are strings, so this table is not weak.
> It is not a "cache" in the sense that it is allowed to drop things. Rather
> it is a registry of module instances. Only a registry as a whole can be
> gc'ed, and which point that context is no longer around for instantiating
> or reinstantiating modules.
> >
> > As you suggest, if it could drop things because of GC that it would then
> need to regenerate, that would expose the non-determinism of gc. That would
> be a big deal. We carefully designed WeakMaps so that gc was
> non-observable. WeakMaps introduce no observable non-determinism. WeakRefs
> alone expose the non-determinism of gc, and are kept well quarantined from
> the rest of the language.
> >
> >
> > On Tue, Jun 30, 2020 at 5:42 PM #!/JoePea  wrote:
> >>
> >> I am curious: can modules be garbage collected if the exports are not
> >> references by anything anymore? And if so, will the module be
> >> re-evaluated the next time it is imported?
> >>
> >> I haven't tried an experiment to answer this yet. I'll be back to post
> >> findings if someone doesn't post an official answer first.
> >>
> >> I'm thinking about code longevity. For example, if we make
> >> long-running web-based applications with many routes and features (for
> >> sake of example imagine a desktop environment, or a MMORPG game, with
> >> apps or components that are loaded within the same context). Over
> >> time, if imports are not collected, then it means we have a memory
> >> leak.
> >>
> >> Imagine, for example, an infinite-universe MMORPG where you can land
> >> on different planets where the code for features of a planet are
> >> provided by third parties as ES Modules. I know, this might not be a
> >> safe idea to import any code into an app, but just imagine it for sake
> >> of example (imagine we have a continuous integration system to test
> >> and verify code security, or something, before that code is allowed to
> >> be consumed in the app). Imagine you play this app for many many days,
> >> and visit many places, and you leave the app running the whole time
> >> (because farming for resources is disabled if the app is not running,
> >> or something).
> >>
> >> I would imagine that we want unused modules (when we leave a planet,
> >> for example) to be (destroyed) garbage collected so that we don't
> >> waste memory.
> >>
> >> #!/JoePea
> >> ___
> >> es-discuss mailing list
> >> es-discuss@mozilla.org
> >> https://mail.mozilla.org/listinfo/es-discuss
> >
> >
> >
> > --
> >   Cheers,
> >   --MarkM
>


-- 
  Cheers,
  --MarkM
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[#!/JoePea]

How can we ensure that long-running applications (even if theoretical),
that may load and unload an unlimited number of new modules over time
(f.e. routes in a web page specified by 3rd parties as time
progresses), not leak memory?

Even if it is theoretical, I don't like the thought of something that
only ever allocates memory that will never be freed.

Is someone working on a solution for this?


#!/JoePea

On Wed, Jul 1, 2020 at 6:16 AM Mark S. Miller  wrote:
>
> No, definitely not. The table from specifiers to module instances is indexed 
> by specifiers. Specifiers are strings, so this table is not weak. It is not a 
> "cache" in the sense that it is allowed to drop things. Rather it is a 
> registry of module instances. Only a registry as a whole can be gc'ed, and 
> which point that context is no longer around for instantiating or 
> reinstantiating modules.
>
> As you suggest, if it could drop things because of GC that it would then need 
> to regenerate, that would expose the non-determinism of gc. That would be a 
> big deal. We carefully designed WeakMaps so that gc was non-observable. 
> WeakMaps introduce no observable non-determinism. WeakRefs alone expose the 
> non-determinism of gc, and are kept well quarantined from the rest of the 
> language.
>
>
> On Tue, Jun 30, 2020 at 5:42 PM #!/JoePea  wrote:
>>
>> I am curious: can modules be garbage collected if the exports are not
>> references by anything anymore? And if so, will the module be
>> re-evaluated the next time it is imported?
>>
>> I haven't tried an experiment to answer this yet. I'll be back to post
>> findings if someone doesn't post an official answer first.
>>
>> I'm thinking about code longevity. For example, if we make
>> long-running web-based applications with many routes and features (for
>> sake of example imagine a desktop environment, or a MMORPG game, with
>> apps or components that are loaded within the same context). Over
>> time, if imports are not collected, then it means we have a memory
>> leak.
>>
>> Imagine, for example, an infinite-universe MMORPG where you can land
>> on different planets where the code for features of a planet are
>> provided by third parties as ES Modules. I know, this might not be a
>> safe idea to import any code into an app, but just imagine it for sake
>> of example (imagine we have a continuous integration system to test
>> and verify code security, or something, before that code is allowed to
>> be consumed in the app). Imagine you play this app for many many days,
>> and visit many places, and you leave the app running the whole time
>> (because farming for resources is disabled if the app is not running,
>> or something).
>>
>> I would imagine that we want unused modules (when we leave a planet,
>> for example) to be (destroyed) garbage collected so that we don't
>> waste memory.
>>
>> #!/JoePea
>> ___
>> es-discuss mailing list
>> es-discuss@mozilla.org
>> https://mail.mozilla.org/listinfo/es-discuss
>
>
>
> --
>   Cheers,
>   --MarkM
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Mark S. Miller]

No, definitely not. The table from specifiers to module instances is
indexed by specifiers. Specifiers are strings, so this table is not weak.
It is not a "cache" in the sense that it is allowed to drop things. Rather
it is a registry of module instances. Only a registry as a whole can be
gc'ed, and which point that context is no longer around for instantiating
or reinstantiating modules.

As you suggest, if it could drop things because of GC that it would then
need to regenerate, that would expose the non-determinism of gc. That would
be a big deal. We carefully designed WeakMaps so that gc was
non-observable. WeakMaps introduce no observable non-determinism. WeakRefs
alone expose the non-determinism of gc, and are kept well quarantined from
the rest of the language.


On Tue, Jun 30, 2020 at 5:42 PM #!/JoePea  wrote:

> I am curious: can modules be garbage collected if the exports are not
> references by anything anymore? And if so, will the module be
> re-evaluated the next time it is imported?
>
> I haven't tried an experiment to answer this yet. I'll be back to post
> findings if someone doesn't post an official answer first.
>
> I'm thinking about code longevity. For example, if we make
> long-running web-based applications with many routes and features (for
> sake of example imagine a desktop environment, or a MMORPG game, with
> apps or components that are loaded within the same context). Over
> time, if imports are not collected, then it means we have a memory
> leak.
>
> Imagine, for example, an infinite-universe MMORPG where you can land
> on different planets where the code for features of a planet are
> provided by third parties as ES Modules. I know, this might not be a
> safe idea to import any code into an app, but just imagine it for sake
> of example (imagine we have a continuous integration system to test
> and verify code security, or something, before that code is allowed to
> be consumed in the app). Imagine you play this app for many many days,
> and visit many places, and you leave the app running the whole time
> (because farming for resources is disabled if the app is not running,
> or something).
>
> I would imagine that we want unused modules (when we leave a planet,
> for example) to be (destroyed) garbage collected so that we don't
> waste memory.
>
> #!/JoePea
> ___
> es-discuss mailing list
> es-discuss@mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss
>


-- 
  Cheers,
  --MarkM
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Isiah Meadows]

That's part of the caching I'm referring to. And if the cache entry
for it has been evicted, I would *not* expect it to necessarily return
the same instance, consistent with the behavior with `require` and
`require.cache` in Node (and similar with most other module loaders
that support cache eviction).

-

Isiah Meadows
cont...@isiahmeadows.com
www.isiahmeadows.com

On Tue, Jun 30, 2020 at 11:57 PM Andrea Giammarchi
 wrote:
>
> even if dereferenced, a dynamic import could re-reference it any time, and I 
> would expect it to still be the same module, it'd be a surprise otherwise 
> (cached things, same namespace checks, etc).
>
> On Wed, Jul 1, 2020 at 7:33 AM Isiah Meadows  wrote:
>>
>> Just to expand on that, if the module record itself is dereferenced
>> (like if it's evicted from the cache somehow), then yes, it should be
>> collected as appropriate. However, I'm not aware of any major
>> implementation that offers that functionality.
>>
>> -
>>
>> Isiah Meadows
>> cont...@isiahmeadows.com
>> www.isiahmeadows.com
>>
>> On Tue, Jun 30, 2020 at 6:22 PM Gus Caplan  wrote:
>> >
>> > Modules in the spec are cached by specifier by modules that import them. 
>> > Modules in major implementations are additionally cached for the entire 
>> > realm by absolute URLs. I would say that for actual code (functions and 
>> > classes and whatnot) leaks aren't really a problem. Even if you import a 
>> > ton of levels, that's not that much memory. The main concern I've seen 
>> > raised is JSON modules, where once you import them the JSON object, which 
>> > can be quite large, will never be collected. Of course, there is a simple 
>> > solution to that (fetch) so it isn't a world ending problem.
>> >
>> > On Tue, Jun 30, 2020 at 7:41 PM #!/JoePea  wrote:
>> >>
>> >> I am curious: can modules be garbage collected if the exports are not
>> >> references by anything anymore? And if so, will the module be
>> >> re-evaluated the next time it is imported?
>> >>
>> >> I haven't tried an experiment to answer this yet. I'll be back to post
>> >> findings if someone doesn't post an official answer first.
>> >>
>> >> I'm thinking about code longevity. For example, if we make
>> >> long-running web-based applications with many routes and features (for
>> >> sake of example imagine a desktop environment, or a MMORPG game, with
>> >> apps or components that are loaded within the same context). Over
>> >> time, if imports are not collected, then it means we have a memory
>> >> leak.
>> >>
>> >> Imagine, for example, an infinite-universe MMORPG where you can land
>> >> on different planets where the code for features of a planet are
>> >> provided by third parties as ES Modules. I know, this might not be a
>> >> safe idea to import any code into an app, but just imagine it for sake
>> >> of example (imagine we have a continuous integration system to test
>> >> and verify code security, or something, before that code is allowed to
>> >> be consumed in the app). Imagine you play this app for many many days,
>> >> and visit many places, and you leave the app running the whole time
>> >> (because farming for resources is disabled if the app is not running,
>> >> or something).
>> >>
>> >> I would imagine that we want unused modules (when we leave a planet,
>> >> for example) to be (destroyed) garbage collected so that we don't
>> >> waste memory.
>> >>
>> >> #!/JoePea
>> >> ___
>> >> es-discuss mailing list
>> >> es-discuss@mozilla.org
>> >> https://mail.mozilla.org/listinfo/es-discuss
>> >
>> > ___
>> > es-discuss mailing list
>> > es-discuss@mozilla.org
>> > https://mail.mozilla.org/listinfo/es-discuss
>> ___
>> es-discuss mailing list
>> es-discuss@mozilla.org
>> https://mail.mozilla.org/listinfo/es-discuss
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Andrea Giammarchi]

even if dereferenced, a dynamic import could re-reference it any time, and
I would expect it to still be the same module, it'd be a surprise otherwise
(cached things, same namespace checks, etc).

On Wed, Jul 1, 2020 at 7:33 AM Isiah Meadows 
wrote:

> Just to expand on that, if the module record itself is dereferenced
> (like if it's evicted from the cache somehow), then yes, it should be
> collected as appropriate. However, I'm not aware of any major
> implementation that offers that functionality.
>
> -
>
> Isiah Meadows
> cont...@isiahmeadows.com
> www.isiahmeadows.com
>
> On Tue, Jun 30, 2020 at 6:22 PM Gus Caplan  wrote:
> >
> > Modules in the spec are cached by specifier by modules that import them.
> Modules in major implementations are additionally cached for the entire
> realm by absolute URLs. I would say that for actual code (functions and
> classes and whatnot) leaks aren't really a problem. Even if you import a
> ton of levels, that's not that much memory. The main concern I've seen
> raised is JSON modules, where once you import them the JSON object, which
> can be quite large, will never be collected. Of course, there is a simple
> solution to that (fetch) so it isn't a world ending problem.
> >
> > On Tue, Jun 30, 2020 at 7:41 PM #!/JoePea  wrote:
> >>
> >> I am curious: can modules be garbage collected if the exports are not
> >> references by anything anymore? And if so, will the module be
> >> re-evaluated the next time it is imported?
> >>
> >> I haven't tried an experiment to answer this yet. I'll be back to post
> >> findings if someone doesn't post an official answer first.
> >>
> >> I'm thinking about code longevity. For example, if we make
> >> long-running web-based applications with many routes and features (for
> >> sake of example imagine a desktop environment, or a MMORPG game, with
> >> apps or components that are loaded within the same context). Over
> >> time, if imports are not collected, then it means we have a memory
> >> leak.
> >>
> >> Imagine, for example, an infinite-universe MMORPG where you can land
> >> on different planets where the code for features of a planet are
> >> provided by third parties as ES Modules. I know, this might not be a
> >> safe idea to import any code into an app, but just imagine it for sake
> >> of example (imagine we have a continuous integration system to test
> >> and verify code security, or something, before that code is allowed to
> >> be consumed in the app). Imagine you play this app for many many days,
> >> and visit many places, and you leave the app running the whole time
> >> (because farming for resources is disabled if the app is not running,
> >> or something).
> >>
> >> I would imagine that we want unused modules (when we leave a planet,
> >> for example) to be (destroyed) garbage collected so that we don't
> >> waste memory.
> >>
> >> #!/JoePea
> >> ___
> >> es-discuss mailing list
> >> es-discuss@mozilla.org
> >> https://mail.mozilla.org/listinfo/es-discuss
> >
> > ___
> > es-discuss mailing list
> > es-discuss@mozilla.org
> > https://mail.mozilla.org/listinfo/es-discuss
> ___
> es-discuss mailing list
> es-discuss@mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss
>
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Isiah Meadows]

Just to expand on that, if the module record itself is dereferenced
(like if it's evicted from the cache somehow), then yes, it should be
collected as appropriate. However, I'm not aware of any major
implementation that offers that functionality.

-

Isiah Meadows
cont...@isiahmeadows.com
www.isiahmeadows.com

On Tue, Jun 30, 2020 at 6:22 PM Gus Caplan  wrote:
>
> Modules in the spec are cached by specifier by modules that import them. 
> Modules in major implementations are additionally cached for the entire realm 
> by absolute URLs. I would say that for actual code (functions and classes and 
> whatnot) leaks aren't really a problem. Even if you import a ton of levels, 
> that's not that much memory. The main concern I've seen raised is JSON 
> modules, where once you import them the JSON object, which can be quite 
> large, will never be collected. Of course, there is a simple solution to that 
> (fetch) so it isn't a world ending problem.
>
> On Tue, Jun 30, 2020 at 7:41 PM #!/JoePea  wrote:
>>
>> I am curious: can modules be garbage collected if the exports are not
>> references by anything anymore? And if so, will the module be
>> re-evaluated the next time it is imported?
>>
>> I haven't tried an experiment to answer this yet. I'll be back to post
>> findings if someone doesn't post an official answer first.
>>
>> I'm thinking about code longevity. For example, if we make
>> long-running web-based applications with many routes and features (for
>> sake of example imagine a desktop environment, or a MMORPG game, with
>> apps or components that are loaded within the same context). Over
>> time, if imports are not collected, then it means we have a memory
>> leak.
>>
>> Imagine, for example, an infinite-universe MMORPG where you can land
>> on different planets where the code for features of a planet are
>> provided by third parties as ES Modules. I know, this might not be a
>> safe idea to import any code into an app, but just imagine it for sake
>> of example (imagine we have a continuous integration system to test
>> and verify code security, or something, before that code is allowed to
>> be consumed in the app). Imagine you play this app for many many days,
>> and visit many places, and you leave the app running the whole time
>> (because farming for resources is disabled if the app is not running,
>> or something).
>>
>> I would imagine that we want unused modules (when we leave a planet,
>> for example) to be (destroyed) garbage collected so that we don't
>> waste memory.
>>
>> #!/JoePea
>> ___
>> es-discuss mailing list
>> es-discuss@mozilla.org
>> https://mail.mozilla.org/listinfo/es-discuss
>
> ___
> es-discuss mailing list
> es-discuss@mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss

Re: Are ES Modules garbage collected? If so, do they re-execute on next import?

[Gus Caplan]

Modules in the spec are cached by specifier by modules that import them.
Modules in major implementations are additionally cached for the entire
realm by absolute URLs. I would say that for actual code (functions and
classes and whatnot) leaks aren't really a problem. Even if you import a
ton of levels, that's not that much memory. The main concern I've seen
raised is JSON modules, where once you import them the JSON object, which
can be quite large, will never be collected. Of course, there is a simple
solution to that (fetch) so it isn't a world ending problem.

On Tue, Jun 30, 2020 at 7:41 PM #!/JoePea  wrote:

> I am curious: can modules be garbage collected if the exports are not
> references by anything anymore? And if so, will the module be
> re-evaluated the next time it is imported?
>
> I haven't tried an experiment to answer this yet. I'll be back to post
> findings if someone doesn't post an official answer first.
>
> I'm thinking about code longevity. For example, if we make
> long-running web-based applications with many routes and features (for
> sake of example imagine a desktop environment, or a MMORPG game, with
> apps or components that are loaded within the same context). Over
> time, if imports are not collected, then it means we have a memory
> leak.
>
> Imagine, for example, an infinite-universe MMORPG where you can land
> on different planets where the code for features of a planet are
> provided by third parties as ES Modules. I know, this might not be a
> safe idea to import any code into an app, but just imagine it for sake
> of example (imagine we have a continuous integration system to test
> and verify code security, or something, before that code is allowed to
> be consumed in the app). Imagine you play this app for many many days,
> and visit many places, and you leave the app running the whole time
> (because farming for resources is disabled if the app is not running,
> or something).
>
> I would imagine that we want unused modules (when we leave a planet,
> for example) to be (destroyed) garbage collected so that we don't
> waste memory.
>
> #!/JoePea
> ___
> es-discuss mailing list
> es-discuss@mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss
>
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss