Re: Weak callbacks?
On 11/06/2013 07:37 PM, K. Gadd wrote: Generally speaking, all the discussions on this list about WRs so far have not suggested that there is any way to introduce WeakRefs without making GC behavior observable in some fashion. WeakRefs functionally make GC observable because when you try to get at the target of the weakref, you either get it or you don't. Once you introduce the ability to get back 'null' when asking for the target of a WR (or any similar failure case), you can use that to determine things about GC behavior. If you somehow eliminate this weakness, WRs no longer solve the problems they are intended to solve. Or at the least, they solve only a small reduced subset of the problems solved in real software via the use of weak references. Being able to access the target of the WR (and have this operation fail) is core to their value (even if finalization notifications are also used quite often). I've already argued in the past about why weak references are important, and why not having solutions for those problems will kneecap the development of web applications by either preventing certain types of software from running in the browser, or forcing implementers to write their own GCs (or entire runtime environments) inside the browser, as is done with environments like emscripten and native client. Once that becomes the solution, the ES spec is irrelevant for those applications because they have had to abandon the language. While the risk of something like this happening is still relatively low, the risk increases over time as more people begin seriously considering solutions like emscripten and nacl - we're starting to see companies ship real products using them already. If this spreads to popular reusable libraries (physics libraries, rendering libraries, etc), there's a risk that those libraries will pull new applications out of the ES realm because it's not possible to use those libraries without abandoning ES in favor of a custom GC/runtime environment. Based on the conversations thus far, a choice just has to be made between the two downsides: exposing some amount of GC internals, or making it impossible to write some subset of applications in ES. It's possible that exposing GC semantics has such catastrophic security consequences that ruling those applications out is merited. It's also possible that workarounds can be applied to reduce the harm of GC visibility (I think in the past someone - maybe Mark? - suggested that disabling cross-realm WRs would mitigate the damage considerably?) This is a false dichotomy. At the extreme, we could simply ship a new builtin resource manager which has it's own GC behaviour that we can expose at will. Given that the sorts of resources that people want to use the memory GC to manage generally have very different cost and volume tradeoffs than memory [1], this is actually much more reasonable than it sounds. The real problem with weak things is that they do have a performance impact on the GC, even when not used. Missing weak-maps can at least be worked around; a slow environment cannot. -Terrence 1 - http://www.mail-archive.com/dev-tech-js-engine-internals@lists.mozilla.org/msg00572.html On Wed, Nov 6, 2013 at 3:23 PM, Oliver Hunt oli...@apple.com wrote: On Nov 6, 2013, at 3:14 PM, Rick Waldron waldron.r...@gmail.com wrote: On Wed, Nov 6, 2013 at 2:15 PM, Domenic Denicola dome...@domenicdenicola.com wrote: Thanks Mark for the education, especially on the pre- vs. post-morterm finalization distinction. I don't think I was specifically advocating for pre-mortem in my OP, since I didn't really understand the difference :P. Post-mortem finalization sounds quite reasonable. What do people think of introducing it into ECMAScript? This may be a naïve question, but how would the handler know which object/weakref had been gc'ed? You wouldn’t :) I’m kind of anti-finalisers in JS for all of the reasons people have raised - they are extremely hazardous, expose non-deterministic behaviour, etc Given our general desire to avoid exposing internal GC semantics, and the difficulty in defining observable GC behaviour (I suspect this would be a non-starter in any case), I can’t see any specification that would allow useful finalisers or WeakRefs. If MarkM has an idea for WeakRefs that don’t leak observable GC behaviour i’d love to hear, but in general i’m opposed to both them and finalisers :-/ —Oliver Rick ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Weak callbacks?
When I talk about the importance of weak references I am *not* talking about the importance of finalization for managing native/remote resources; that's an important consideration but it's a separate one. When I talk about why weak references are important, I mean for the purposes of building large applications without creating uncollectable cycles in object graphs. Doing this without weak references is *very* difficult in some scenarios, and in other scenarios requires manual management of all object lifetimes (addref/release style, or having a single lifetime owner that 'deletes' an object on demand). That's the sort of obstacle that factors into a developer's choice of language and toolset. I've seen this particular concern with ES crop up in the past on real projects, and I've seen firsthand how difficult it is to avoid uncollectable cycles in a language environment without any sort of weak reference mechanism. Leaking large uncollectable cycles can have catastrophic consequences in multimedia applications and games, where those cycles might be retaining images or sounds or other huge game assets. On Thu, Nov 7, 2013 at 9:19 AM, Terrence Cole tc...@mozilla.com wrote: On 11/06/2013 07:37 PM, K. Gadd wrote: Generally speaking, all the discussions on this list about WRs so far have not suggested that there is any way to introduce WeakRefs without making GC behavior observable in some fashion. WeakRefs functionally make GC observable because when you try to get at the target of the weakref, you either get it or you don't. Once you introduce the ability to get back 'null' when asking for the target of a WR (or any similar failure case), you can use that to determine things about GC behavior. If you somehow eliminate this weakness, WRs no longer solve the problems they are intended to solve. Or at the least, they solve only a small reduced subset of the problems solved in real software via the use of weak references. Being able to access the target of the WR (and have this operation fail) is core to their value (even if finalization notifications are also used quite often). I've already argued in the past about why weak references are important, and why not having solutions for those problems will kneecap the development of web applications by either preventing certain types of software from running in the browser, or forcing implementers to write their own GCs (or entire runtime environments) inside the browser, as is done with environments like emscripten and native client. Once that becomes the solution, the ES spec is irrelevant for those applications because they have had to abandon the language. While the risk of something like this happening is still relatively low, the risk increases over time as more people begin seriously considering solutions like emscripten and nacl - we're starting to see companies ship real products using them already. If this spreads to popular reusable libraries (physics libraries, rendering libraries, etc), there's a risk that those libraries will pull new applications out of the ES realm because it's not possible to use those libraries without abandoning ES in favor of a custom GC/runtime environment. Based on the conversations thus far, a choice just has to be made between the two downsides: exposing some amount of GC internals, or making it impossible to write some subset of applications in ES. It's possible that exposing GC semantics has such catastrophic security consequences that ruling those applications out is merited. It's also possible that workarounds can be applied to reduce the harm of GC visibility (I think in the past someone - maybe Mark? - suggested that disabling cross-realm WRs would mitigate the damage considerably?) This is a false dichotomy. At the extreme, we could simply ship a new builtin resource manager which has it's own GC behaviour that we can expose at will. Given that the sorts of resources that people want to use the memory GC to manage generally have very different cost and volume tradeoffs than memory [1], this is actually much more reasonable than it sounds. The real problem with weak things is that they do have a performance impact on the GC, even when not used. Missing weak-maps can at least be worked around; a slow environment cannot. -Terrence 1 - http://www.mail-archive.com/dev-tech-js-engine-internals@lists.mozilla.org/msg00572.html On Wed, Nov 6, 2013 at 3:23 PM, Oliver Hunt oli...@apple.com wrote: On Nov 6, 2013, at 3:14 PM, Rick Waldron waldron.r...@gmail.com wrote: On Wed, Nov 6, 2013 at 2:15 PM, Domenic Denicola dome...@domenicdenicola.com wrote: Thanks Mark for the education, especially on the pre- vs. post-morterm finalization distinction. I don't think I was specifically advocating for pre-mortem in my OP, since I didn't really understand the difference :P. Post-mortem finalization
Re: Weak callbacks?
Could you give an example, hopefully simple, of such a problem you can solve better with GC + weakmaps + weakrefs but without finalization, vs just GC + weakmaps ? Thanks. On Thu, Nov 7, 2013 at 1:46 PM, K. Gadd k...@luminance.org wrote: When I talk about the importance of weak references I am *not* talking about the importance of finalization for managing native/remote resources; that's an important consideration but it's a separate one. When I talk about why weak references are important, I mean for the purposes of building large applications without creating uncollectable cycles in object graphs. Doing this without weak references is *very* difficult in some scenarios, and in other scenarios requires manual management of all object lifetimes (addref/release style, or having a single lifetime owner that 'deletes' an object on demand). That's the sort of obstacle that factors into a developer's choice of language and toolset. I've seen this particular concern with ES crop up in the past on real projects, and I've seen firsthand how difficult it is to avoid uncollectable cycles in a language environment without any sort of weak reference mechanism. Leaking large uncollectable cycles can have catastrophic consequences in multimedia applications and games, where those cycles might be retaining images or sounds or other huge game assets. On Thu, Nov 7, 2013 at 9:19 AM, Terrence Cole tc...@mozilla.com wrote: On 11/06/2013 07:37 PM, K. Gadd wrote: Generally speaking, all the discussions on this list about WRs so far have not suggested that there is any way to introduce WeakRefs without making GC behavior observable in some fashion. WeakRefs functionally make GC observable because when you try to get at the target of the weakref, you either get it or you don't. Once you introduce the ability to get back 'null' when asking for the target of a WR (or any similar failure case), you can use that to determine things about GC behavior. If you somehow eliminate this weakness, WRs no longer solve the problems they are intended to solve. Or at the least, they solve only a small reduced subset of the problems solved in real software via the use of weak references. Being able to access the target of the WR (and have this operation fail) is core to their value (even if finalization notifications are also used quite often). I've already argued in the past about why weak references are important, and why not having solutions for those problems will kneecap the development of web applications by either preventing certain types of software from running in the browser, or forcing implementers to write their own GCs (or entire runtime environments) inside the browser, as is done with environments like emscripten and native client. Once that becomes the solution, the ES spec is irrelevant for those applications because they have had to abandon the language. While the risk of something like this happening is still relatively low, the risk increases over time as more people begin seriously considering solutions like emscripten and nacl - we're starting to see companies ship real products using them already. If this spreads to popular reusable libraries (physics libraries, rendering libraries, etc), there's a risk that those libraries will pull new applications out of the ES realm because it's not possible to use those libraries without abandoning ES in favor of a custom GC/runtime environment. Based on the conversations thus far, a choice just has to be made between the two downsides: exposing some amount of GC internals, or making it impossible to write some subset of applications in ES. It's possible that exposing GC semantics has such catastrophic security consequences that ruling those applications out is merited. It's also possible that workarounds can be applied to reduce the harm of GC visibility (I think in the past someone - maybe Mark? - suggested that disabling cross-realm WRs would mitigate the damage considerably?) This is a false dichotomy. At the extreme, we could simply ship a new builtin resource manager which has it's own GC behaviour that we can expose at will. Given that the sorts of resources that people want to use the memory GC to manage generally have very different cost and volume tradeoffs than memory [1], this is actually much more reasonable than it sounds. The real problem with weak things is that they do have a performance impact on the GC, even when not used. Missing weak-maps can at least be worked around; a slow environment cannot. -Terrence 1 - http://www.mail-archive.com/dev-tech-js-engine-internals@lists.mozilla.org/msg00572.html On Wed, Nov 6, 2013 at 3:23 PM, Oliver Hunt oli...@apple.com wrote: On Nov 6, 2013, at 3:14 PM, Rick Waldron waldron.r...@gmail.com wrote: On Wed, Nov 6, 2013 at 2:15 PM, Domenic Denicola dome...@domenicdenicola.com
Re: Weak callbacks?
I'll try and restate an example I used before. Let's say you're building a somewhat simple tab-based application, where each tab represents a different 'mode'. In this case, you need a higher level coordinating object that manages the tab bar, and it needs to own references to all of the active modes in order to manage the tabs. Similarly, each mode may need to have a reference to the coordinator in order to interact with it - like by setting the title of the tab, or opening a new tab containing another mode - and it may need to have references to other modes in order to communicate (like if one tab opens a tab containing related information). In a model like that, you can trivially end up with object cycles. All the references (going in every direction) are strong, so to successfully collect a mode you need to ensure that every reference to it from live objects is released. This is problematic because it means that, for example: If Mode A opens a new tab containing Mode B, and Mode A wants to respond to things happening in Mode B, Mode A now has to register some sort of event listener on Mode B. This means that Mode A and Mode B have now formed a strong cycle: Mode B's event listener list contains a strong reference to Mode A (to fire the event notification), and Mode A has a strong reference to Mode B in order to be able to respond to those events. Now, if you close Mode B, the coordinator gets rid of the tab and drops its strong references to Mode B - the tab is 'gone' - but there's no trivial way to ensure that everyone else holding a reference to it is dead. If multiple tabs interact in this way it's possible for a huge graph of objects to be hanging off one live tab. Eventually, if you close enough tabs you might 'free' the graph, but if you aren't careful the coordinator itself can end up keeping dead tabs alive, with things like event listener lists. In the real apps I've worked on like this, the solution was some form of weak references - making event listeners own weak self-references instead of strong self-references, so that in normal circumstances event subscriptions expire along with the subscriber, along with using weak references directly in cases like 'tab a spawns tab b' in order to ensure that you aren't keeping an object alive just to monitor it. IIRC, cycles like this are already common in some DOM scenarios, where event subscription and such result in big graphs of objects hanging off a single live object? I've certainly run into it in a few cases, where an event listener hanging off an audio element caused a huge graph of objects to leak, and I had to manually remove the event listeners at an appropriate time (and finding an appropriate time can be difficult!) On Thu, Nov 7, 2013 at 2:33 PM, Mark S. Miller erig...@google.com wrote: Could you give an example, hopefully simple, of such a problem you can solve better with GC + weakmaps + weakrefs but without finalization, vs just GC + weakmaps ? Thanks. On Thu, Nov 7, 2013 at 1:46 PM, K. Gadd k...@luminance.org wrote: When I talk about the importance of weak references I am *not* talking about the importance of finalization for managing native/remote resources; that's an important consideration but it's a separate one. When I talk about why weak references are important, I mean for the purposes of building large applications without creating uncollectable cycles in object graphs. Doing this without weak references is *very* difficult in some scenarios, and in other scenarios requires manual management of all object lifetimes (addref/release style, or having a single lifetime owner that 'deletes' an object on demand). That's the sort of obstacle that factors into a developer's choice of language and toolset. I've seen this particular concern with ES crop up in the past on real projects, and I've seen firsthand how difficult it is to avoid uncollectable cycles in a language environment without any sort of weak reference mechanism. Leaking large uncollectable cycles can have catastrophic consequences in multimedia applications and games, where those cycles might be retaining images or sounds or other huge game assets. On Thu, Nov 7, 2013 at 9:19 AM, Terrence Cole tc...@mozilla.com wrote: On 11/06/2013 07:37 PM, K. Gadd wrote: Generally speaking, all the discussions on this list about WRs so far have not suggested that there is any way to introduce WeakRefs without making GC behavior observable in some fashion. WeakRefs functionally make GC observable because when you try to get at the target of the weakref, you either get it or you don't. Once you introduce the ability to get back 'null' when asking for the target of a WR (or any similar failure case), you can use that to determine things about GC behavior. If you somehow eliminate this weakness, WRs no longer solve the problems they are intended to solve. Or at the least, they solve only a small reduced subset of the
Re: Weak callbacks?
That example looks to me like it could be implemented with WeakMap instead of WeakRef. Am I missing something? On Thu Nov 07 2013 at 2:43:45 PM, K. Gadd k...@luminance.org wrote: I'll try and restate an example I used before. Let's say you're building a somewhat simple tab-based application, where each tab represents a different 'mode'. In this case, you need a higher level coordinating object that manages the tab bar, and it needs to own references to all of the active modes in order to manage the tabs. Similarly, each mode may need to have a reference to the coordinator in order to interact with it - like by setting the title of the tab, or opening a new tab containing another mode - and it may need to have references to other modes in order to communicate (like if one tab opens a tab containing related information). In a model like that, you can trivially end up with object cycles. All the references (going in every direction) are strong, so to successfully collect a mode you need to ensure that every reference to it from live objects is released. This is problematic because it means that, for example: If Mode A opens a new tab containing Mode B, and Mode A wants to respond to things happening in Mode B, Mode A now has to register some sort of event listener on Mode B. This means that Mode A and Mode B have now formed a strong cycle: Mode B's event listener list contains a strong reference to Mode A (to fire the event notification), and Mode A has a strong reference to Mode B in order to be able to respond to those events. Now, if you close Mode B, the coordinator gets rid of the tab and drops its strong references to Mode B - the tab is 'gone' - but there's no trivial way to ensure that everyone else holding a reference to it is dead. If multiple tabs interact in this way it's possible for a huge graph of objects to be hanging off one live tab. Eventually, if you close enough tabs you might 'free' the graph, but if you aren't careful the coordinator itself can end up keeping dead tabs alive, with things like event listener lists. In the real apps I've worked on like this, the solution was some form of weak references - making event listeners own weak self-references instead of strong self-references, so that in normal circumstances event subscriptions expire along with the subscriber, along with using weak references directly in cases like 'tab a spawns tab b' in order to ensure that you aren't keeping an object alive just to monitor it. IIRC, cycles like this are already common in some DOM scenarios, where event subscription and such result in big graphs of objects hanging off a single live object? I've certainly run into it in a few cases, where an event listener hanging off an audio element caused a huge graph of objects to leak, and I had to manually remove the event listeners at an appropriate time (and finding an appropriate time can be difficult!) On Thu, Nov 7, 2013 at 2:33 PM, Mark S. Miller erig...@google.com wrote: Could you give an example, hopefully simple, of such a problem you can solve better with GC + weakmaps + weakrefs but without finalization, vs just GC + weakmaps ? Thanks. On Thu, Nov 7, 2013 at 1:46 PM, K. Gadd k...@luminance.org wrote: When I talk about the importance of weak references I am *not* talking about the importance of finalization for managing native/remote resources; that's an important consideration but it's a separate one. When I talk about why weak references are important, I mean for the purposes of building large applications without creating uncollectable cycles in object graphs. Doing this without weak references is *very* difficult in some scenarios, and in other scenarios requires manual management of all object lifetimes (addref/release style, or having a single lifetime owner that 'deletes' an object on demand). That's the sort of obstacle that factors into a developer's choice of language and toolset. I've seen this particular concern with ES crop up in the past on real projects, and I've seen firsthand how difficult it is to avoid uncollectable cycles in a language environment without any sort of weak reference mechanism. Leaking large uncollectable cycles can have catastrophic consequences in multimedia applications and games, where those cycles might be retaining images or sounds or other huge game assets. On Thu, Nov 7, 2013 at 9:19 AM, Terrence Cole tc...@mozilla.com wrote: On 11/06/2013 07:37 PM, K. Gadd wrote: Generally speaking, all the discussions on this list about WRs so far have not suggested that there is any way to introduce WeakRefs without making GC behavior observable in some fashion. WeakRefs functionally make GC observable because when you try to get at the target of the weakref, you either get it or you don't. Once you introduce the ability to get back 'null' when asking for the target of a WR (or any similar failure case), you can use
Re: Weak callbacks?
I am not aware of any way to implement my described scenario using WeakMaps without having the same cycle collection issues. I haven't seen any examples of WeakMap used for this in the wild either. On Thu, Nov 7, 2013 at 2:48 PM, felix lee feli...@gmail.com wrote: That example looks to me like it could be implemented with WeakMap instead of WeakRef. Am I missing something? On Thu Nov 07 2013 at 2:43:45 PM, K. Gadd k...@luminance.org wrote: I'll try and restate an example I used before. Let's say you're building a somewhat simple tab-based application, where each tab represents a different 'mode'. In this case, you need a higher level coordinating object that manages the tab bar, and it needs to own references to all of the active modes in order to manage the tabs. Similarly, each mode may need to have a reference to the coordinator in order to interact with it - like by setting the title of the tab, or opening a new tab containing another mode - and it may need to have references to other modes in order to communicate (like if one tab opens a tab containing related information). In a model like that, you can trivially end up with object cycles. All the references (going in every direction) are strong, so to successfully collect a mode you need to ensure that every reference to it from live objects is released. This is problematic because it means that, for example: If Mode A opens a new tab containing Mode B, and Mode A wants to respond to things happening in Mode B, Mode A now has to register some sort of event listener on Mode B. This means that Mode A and Mode B have now formed a strong cycle: Mode B's event listener list contains a strong reference to Mode A (to fire the event notification), and Mode A has a strong reference to Mode B in order to be able to respond to those events. Now, if you close Mode B, the coordinator gets rid of the tab and drops its strong references to Mode B - the tab is 'gone' - but there's no trivial way to ensure that everyone else holding a reference to it is dead. If multiple tabs interact in this way it's possible for a huge graph of objects to be hanging off one live tab. Eventually, if you close enough tabs you might 'free' the graph, but if you aren't careful the coordinator itself can end up keeping dead tabs alive, with things like event listener lists. In the real apps I've worked on like this, the solution was some form of weak references - making event listeners own weak self-references instead of strong self-references, so that in normal circumstances event subscriptions expire along with the subscriber, along with using weak references directly in cases like 'tab a spawns tab b' in order to ensure that you aren't keeping an object alive just to monitor it. IIRC, cycles like this are already common in some DOM scenarios, where event subscription and such result in big graphs of objects hanging off a single live object? I've certainly run into it in a few cases, where an event listener hanging off an audio element caused a huge graph of objects to leak, and I had to manually remove the event listeners at an appropriate time (and finding an appropriate time can be difficult!) On Thu, Nov 7, 2013 at 2:33 PM, Mark S. Miller erig...@google.comwrote: Could you give an example, hopefully simple, of such a problem you can solve better with GC + weakmaps + weakrefs but without finalization, vs just GC + weakmaps ? Thanks. On Thu, Nov 7, 2013 at 1:46 PM, K. Gadd k...@luminance.org wrote: When I talk about the importance of weak references I am *not* talking about the importance of finalization for managing native/remote resources; that's an important consideration but it's a separate one. When I talk about why weak references are important, I mean for the purposes of building large applications without creating uncollectable cycles in object graphs. Doing this without weak references is *very* difficult in some scenarios, and in other scenarios requires manual management of all object lifetimes (addref/release style, or having a single lifetime owner that 'deletes' an object on demand). That's the sort of obstacle that factors into a developer's choice of language and toolset. I've seen this particular concern with ES crop up in the past on real projects, and I've seen firsthand how difficult it is to avoid uncollectable cycles in a language environment without any sort of weak reference mechanism. Leaking large uncollectable cycles can have catastrophic consequences in multimedia applications and games, where those cycles might be retaining images or sounds or other huge game assets. On Thu, Nov 7, 2013 at 9:19 AM, Terrence Cole tc...@mozilla.com wrote: On 11/06/2013 07:37 PM, K. Gadd wrote: Generally speaking, all the discussions on this list about WRs so far have not suggested that there is any way to introduce WeakRefs without making GC behavior observable in some fashion.
Re: Weak callbacks?
Ah, sorry, I misread your description of the relationships. Yeah, that would be hard to do with WeakMaps. On Thu Nov 07 2013 at 3:02:02 PM, K. Gadd k...@luminance.org wrote: I am not aware of any way to implement my described scenario using WeakMaps without having the same cycle collection issues. I haven't seen any examples of WeakMap used for this in the wild either. On Thu, Nov 7, 2013 at 2:48 PM, felix lee feli...@gmail.com wrote: That example looks to me like it could be implemented with WeakMap instead of WeakRef. Am I missing something? On Thu Nov 07 2013 at 2:43:45 PM, K. Gadd k...@luminance.org wrote: I'll try and restate an example I used before. Let's say you're building a somewhat simple tab-based application, where each tab represents a different 'mode'. In this case, you need a higher level coordinating object that manages the tab bar, and it needs to own references to all of the active modes in order to manage the tabs. Similarly, each mode may need to have a reference to the coordinator in order to interact with it - like by setting the title of the tab, or opening a new tab containing another mode - and it may need to have references to other modes in order to communicate (like if one tab opens a tab containing related information). In a model like that, you can trivially end up with object cycles. All the references (going in every direction) are strong, so to successfully collect a mode you need to ensure that every reference to it from live objects is released. This is problematic because it means that, for example: If Mode A opens a new tab containing Mode B, and Mode A wants to respond to things happening in Mode B, Mode A now has to register some sort of event listener on Mode B. This means that Mode A and Mode B have now formed a strong cycle: Mode B's event listener list contains a strong reference to Mode A (to fire the event notification), and Mode A has a strong reference to Mode B in order to be able to respond to those events. Now, if you close Mode B, the coordinator gets rid of the tab and drops its strong references to Mode B - the tab is 'gone' - but there's no trivial way to ensure that everyone else holding a reference to it is dead. If multiple tabs interact in this way it's possible for a huge graph of objects to be hanging off one live tab. Eventually, if you close enough tabs you might 'free' the graph, but if you aren't careful the coordinator itself can end up keeping dead tabs alive, with things like event listener lists. In the real apps I've worked on like this, the solution was some form of weak references - making event listeners own weak self-references instead of strong self-references, so that in normal circumstances event subscriptions expire along with the subscriber, along with using weak references directly in cases like 'tab a spawns tab b' in order to ensure that you aren't keeping an object alive just to monitor it. IIRC, cycles like this are already common in some DOM scenarios, where event subscription and such result in big graphs of objects hanging off a single live object? I've certainly run into it in a few cases, where an event listener hanging off an audio element caused a huge graph of objects to leak, and I had to manually remove the event listeners at an appropriate time (and finding an appropriate time can be difficult!) On Thu, Nov 7, 2013 at 2:33 PM, Mark S. Miller erig...@google.com wrote: Could you give an example, hopefully simple, of such a problem you can solve better with GC + weakmaps + weakrefs but without finalization, vs just GC + weakmaps ? Thanks. On Thu, Nov 7, 2013 at 1:46 PM, K. Gadd k...@luminance.org wrote: When I talk about the importance of weak references I am *not* talking about the importance of finalization for managing native/remote resources; that's an important consideration but it's a separate one. When I talk about why weak references are important, I mean for the purposes of building large applications without creating uncollectable cycles in object graphs. Doing this without weak references is *very* difficult in some scenarios, and in other scenarios requires manual management of all object lifetimes (addref/release style, or having a single lifetime owner that 'deletes' an object on demand). That's the sort of obstacle that factors into a developer's choice of language and toolset. I've seen this particular concern with ES crop up in the past on real projects, and I've seen firsthand how difficult it is to avoid uncollectable cycles in a language environment without any sort of weak reference mechanism. Leaking large uncollectable cycles can have catastrophic consequences in multimedia applications and games, where those cycles might be retaining images or sounds or other huge game assets. On Thu, Nov 7, 2013 at 9:19 AM, Terrence Cole tc...@mozilla.com wrote: On 11/06/2013 07:37 PM, K. Gadd wrote: Generally
Re: Weak callbacks?
Le 07/11/2013 22:46, K. Gadd a écrit : That's the sort of obstacle that factors into a developer's choice of language and toolset. I've seen this particular concern with ES crop up in the past on real projects, and I've seen firsthand how difficult it is to avoid uncollectable cycles in a language environment without any sort of weak reference mechanism. Leaking large uncollectable cycles can have catastrophic consequences in multimedia applications and games, where those cycles might be retaining images or sounds or other huge game assets. The repeated use of the word cycle worries me. Cycles aren't a problem by themselves with mark and sweep, do we agree? David ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Weak callbacks?
The problem is that the cycles remain uncollectable for a very long period of time, even though functionally the references don't need to be strong. On Thu, Nov 7, 2013 at 3:10 PM, David Bruant bruan...@gmail.com wrote: Le 07/11/2013 22:46, K. Gadd a écrit : That's the sort of obstacle that factors into a developer's choice of language and toolset. I've seen this particular concern with ES crop up in the past on real projects, and I've seen firsthand how difficult it is to avoid uncollectable cycles in a language environment without any sort of weak reference mechanism. Leaking large uncollectable cycles can have catastrophic consequences in multimedia applications and games, where those cycles might be retaining images or sounds or other huge game assets. The repeated use of the word cycle worries me. Cycles aren't a problem by themselves with mark and sweep, do we agree? David ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Weak callbacks?
I agree. This is a good use for weakrefs without the need for finalization. Thanks! On Thu, Nov 7, 2013 at 3:12 PM, K. Gadd k...@luminance.org wrote: The problem is that the cycles remain uncollectable for a very long period of time, even though functionally the references don't need to be strong. On Thu, Nov 7, 2013 at 3:10 PM, David Bruant bruan...@gmail.com wrote: Le 07/11/2013 22:46, K. Gadd a écrit : That's the sort of obstacle that factors into a developer's choice of language and toolset. I've seen this particular concern with ES crop up in the past on real projects, and I've seen firsthand how difficult it is to avoid uncollectable cycles in a language environment without any sort of weak reference mechanism. Leaking large uncollectable cycles can have catastrophic consequences in multimedia applications and games, where those cycles might be retaining images or sounds or other huge game assets. The repeated use of the word cycle worries me. Cycles aren't a problem by themselves with mark and sweep, do we agree? David ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss -- Text by me above is hereby placed in the public domain Cheers, --MarkM ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Weak callbacks?
Would someone be interested in putting together a small code example, before and after weak refs, illustrating this situation? I have a feeling this conversation will be referenced in many future threads, and the added clarity of code would be greatly helpful. On 7 Nov 2013, at 18:16, Mark Miller erig...@gmail.commailto:erig...@gmail.com wrote: I agree. This is a good use for weakrefs without the need for finalization. Thanks! On Thu, Nov 7, 2013 at 3:12 PM, K. Gadd k...@luminance.orgmailto:k...@luminance.org wrote: The problem is that the cycles remain uncollectable for a very long period of time, even though functionally the references don't need to be strong. On Thu, Nov 7, 2013 at 3:10 PM, David Bruant bruan...@gmail.commailto:bruan...@gmail.com wrote: Le 07/11/2013 22:46, K. Gadd a ?crit : That's the sort of obstacle that factors into a developer's choice of language and toolset. I've seen this particular concern with ES crop up in the past on real projects, and I've seen firsthand how difficult it is to avoid uncollectable cycles in a language environment without any sort of weak reference mechanism. Leaking large uncollectable cycles can have catastrophic consequences in multimedia applications and games, where those cycles might be retaining images or sounds or other huge game assets. The repeated use of the word cycle worries me. Cycles aren't a problem by themselves with mark and sweep, do we agree? David ___ es-discuss mailing list es-discuss@mozilla.orgmailto:es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss -- Text by me above is hereby placed in the public domain Cheers, --MarkM ___ es-discuss mailing list es-discuss@mozilla.orgmailto:es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Weak callbacks?
On Thu, Nov 7, 2013 at 11:42 PM, K. Gadd k...@luminance.org wrote: I'll try and restate an example I used before. Let's say you're building a somewhat simple tab-based application, where each tab represents a different 'mode'. In this case, you need a higher level coordinating object that manages the tab bar, and it needs to own references to all of the active modes in order to manage the tabs. Similarly, each mode may need to have a reference to the coordinator in order to interact with it - like by setting the title of the tab, or opening a new tab containing another mode - and it may need to have references to other modes in order to communicate (like if one tab opens a tab containing related information). In a model like that, you can trivially end up with object cycles. All the references (going in every direction) are strong, so to successfully collect a mode you need to ensure that every reference to it from live objects is released. This is problematic because it means that, for example: If Mode A opens a new tab containing Mode B, and Mode A wants to respond to things happening in Mode B, Mode A now has to register some sort of event listener on Mode B. This means that Mode A and Mode B have now formed a strong cycle: Mode B's event listener list contains a strong reference to Mode A (to fire the event notification), and Mode A has a strong reference to Mode B in order to be able to respond to those events. Now, if you close Mode B, the coordinator gets rid of the tab and drops its strong references to Mode B - the tab is 'gone' - but there's no trivial way to ensure that everyone else holding a reference to it is dead. If multiple tabs interact in this way it's possible for a huge graph of objects to be hanging off one live tab. Eventually, if you close enough tabs you might 'free' the graph, but if you aren't careful the coordinator itself can end up keeping dead tabs alive, with things like event listener lists. In the real apps I've worked on like this, the solution was some form of weak references - making event listeners own weak self-references instead of strong self-references, so that in normal circumstances event subscriptions expire along with the subscriber, along with using weak references directly in cases like 'tab a spawns tab b' in order to ensure that you aren't keeping an object alive just to monitor it. IIRC, cycles like this are already common in some DOM scenarios, where event subscription and such result in big graphs of objects hanging off a single live object? I've certainly run into it in a few cases, where an event listener hanging off an audio element caused a huge graph of objects to leak, and I had to manually remove the event listeners at an appropriate time (and finding an appropriate time can be difficult!) Instead of using weakRefs to solve your problem you could use weak event listeners. Using weak event listeners would mean you have an event dispatcher which has a weak reference to an event handler. This way the handler object can listen to events from other parts of the system without being kept alive by those other parts. In your example, each tab can be an event handler, and they can weakly listen to events from other tabs. Only the coordinator has a strong reference to the tabs, and so when the coordinator severs the strong reference, the tabs are eventually garbage collected. Or another example would be a page consisting of multiple web components. Each component is strongly reference by the DOM, but they weakly listen to events from each other. This means that you can delete a component by removing it from the DOM. The object will (eventually) be garbage collected, and will no longer listen to events. I have created a gist which describes how weak event listeners could work: https://gist.github.com/mariusGundersen/7364253 It seems that every (real) usecase for weakRefs, weakMaps and weakSets is to implement a weak event listener system. Unfortunately weak event listeners cannot be implemented with the current weakSet/weakMap spec, since they are not iterable. To implement an event dispatch system we need to be able to iterate over the listeners. A weak event dispatcher has a hidden iterable weakSet of listeners. It iterates over this weakSet and notifys each entry of the event which has occured. But it does not expose this iterable set. Does it sound like weak event listeners could solve the challenges you face? Is there a problem which can be implemented with weak references which cannot be solved by weak event listeners? Marius Gundersen On Thu, Nov 7, 2013 at 2:33 PM, Mark S. Miller erig...@google.com wrote: Could you give an example, hopefully simple, of such a problem you can solve better with GC + weakmaps + weakrefs but without finalization, vs just GC + weakmaps ? Thanks. On Thu, Nov 7, 2013 at 1:46 PM, K. Gadd k...@luminance.org wrote: When I talk about the importance of weak
Re: Weak callbacks?
Please reread my example, events are a part of it but it is not exclusively about events. My example explicitly demonstrates the modes holding references in *both* directions, and only one of them is an event listener. You're correct that weak event listeners solve some of the problems that are usually used to justify weakrefs, but they're not the only problems. Sometimes objects really do need to interact bidirectionally without keeping each other alive :) To restate things, part of the problem here is that objects need to interact, so they need to either have references to each other or have a third party act as an intermediary. You can remove references entirely with some sort of message bus or other intermediary (interacting through opaque handles?) but at that point you've just moved all the lifetime management concerns around, and now you're manually calling addref/decref or doing things like that. Alternately, you can keep the strong references and carefully plumb 'release' code through every object that holds references, so that for example when a tab dies, it tells literally every object in the application to let go of any references to the tab. I hope the complexity inherent to *that* method is evident. Apologies if my contrived example is not clear enough for these purposes; the applications I write on a day-to-day basis do not use WRs much. I'm speaking from past experience and the things I hear from users of my compiler, and my past experience was on closed-source projects I can't easily reproduce here verbatim. To contrive a scenario that might illustrate where event listeners aren't necessarily enough: Let's say I have an entity system for a game that is modelling thousands of independent entities that interact. In this system, entity A might decide to chase after entity B, so it grabs a reference to entity B in order to chase after it. Meanwhile, entity B might be chasing after entity C, so it has a reference to entity C. Now, let's say entity C and entity B both are removed from the simulation. At this point, as long as entity A is alive, so are B and C (in GC terms) even though the latter two are not participating in the simulation. Now every place where an entity can refer to another entity has to be manually audited to ensure that 'dead' references are actually nulled out, which means you're basically doing your own GC sweeps over your heap using your own lifetime rules. On Thu, Nov 7, 2013 at 4:27 PM, Marius Gundersen gunder...@gmail.comwrote: On Thu, Nov 7, 2013 at 11:42 PM, K. Gadd k...@luminance.org wrote: I'll try and restate an example I used before. Let's say you're building a somewhat simple tab-based application, where each tab represents a different 'mode'. In this case, you need a higher level coordinating object that manages the tab bar, and it needs to own references to all of the active modes in order to manage the tabs. Similarly, each mode may need to have a reference to the coordinator in order to interact with it - like by setting the title of the tab, or opening a new tab containing another mode - and it may need to have references to other modes in order to communicate (like if one tab opens a tab containing related information). In a model like that, you can trivially end up with object cycles. All the references (going in every direction) are strong, so to successfully collect a mode you need to ensure that every reference to it from live objects is released. This is problematic because it means that, for example: If Mode A opens a new tab containing Mode B, and Mode A wants to respond to things happening in Mode B, Mode A now has to register some sort of event listener on Mode B. This means that Mode A and Mode B have now formed a strong cycle: Mode B's event listener list contains a strong reference to Mode A (to fire the event notification), and Mode A has a strong reference to Mode B in order to be able to respond to those events. Now, if you close Mode B, the coordinator gets rid of the tab and drops its strong references to Mode B - the tab is 'gone' - but there's no trivial way to ensure that everyone else holding a reference to it is dead. If multiple tabs interact in this way it's possible for a huge graph of objects to be hanging off one live tab. Eventually, if you close enough tabs you might 'free' the graph, but if you aren't careful the coordinator itself can end up keeping dead tabs alive, with things like event listener lists. In the real apps I've worked on like this, the solution was some form of weak references - making event listeners own weak self-references instead of strong self-references, so that in normal circumstances event subscriptions expire along with the subscriber, along with using weak references directly in cases like 'tab a spawns tab b' in order to ensure that you aren't keeping an object alive just to monitor it. IIRC, cycles like this are already common in some DOM scenarios,
Re: Weak callbacks?
On Thu, Nov 7, 2013 at 4:27 PM, Marius Gundersen gunder...@gmail.comwrote: It seems that every (real) usecase for weakRefs, weakMaps and weakSets is to implement a weak event listener system. Not true. Even silly. Unfortunately weak event listeners cannot be implemented with the current weakSet/weakMap spec, since they are not iterable. If they were iterable, they would be broken, and fail to solve their actual real use cases. They are not the right tool for the job -- they have nothing to do with weak event listeners. It seems like the only real use case for email is to make toast. Unfortunately, email can't be used to make toast because it isn't hot enough. -- Text by me above is hereby placed in the public domain Cheers, --MarkM ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
Re: Weak callbacks?
On Nov 8, 2013 1:46 AM, Mark Miller erig...@gmail.com wrote: On Thu, Nov 7, 2013 at 4:27 PM, Marius Gundersen gunder...@gmail.com wrote: It seems that every (real) usecase for weakRefs, weakMaps and weakSets is to implement a weak event listener system. Not true. Even silly. Sorry, I may have worded that a bit strongly Unfortunately weak event listeners cannot be implemented with the current weakSet/weakMap spec, since they are not iterable. If they were iterable, they would be broken, and fail to solve their actual real use cases. They are not the right tool for the job -- they have nothing to do with weak event listeners. Yes, that was my point. Weak event listeners should be implemented in the browser to solve the kind of problems brought up towards the end of this thread. I never meant to say that weak sets or maps should be used to implement weak event listeners. Many common problems can be solved with weak event listeners, and so they should be standardized and implemented in the language. It seems like the only real use case for email is to make toast. Unfortunately, email can't be used to make toast because it isn't hot enough. -- Text by me above is hereby placed in the public domain Cheers, --MarkM ___ es-discuss mailing list es-discuss@mozilla.org https://mail.mozilla.org/listinfo/es-discuss
