Fwd: Removal of WeakMap/WeakSet clear

2014-12-09 Thread Mark S. Miller
[Katelyn, I decided to share this challenge publicly] I very much respect experience-driven feedback [...]. [For any of you with an implementation based use case for WeakCollections, do] you have, or can you cook up based on what you have, a realistic performance critical use of WeakMap with

Re: Removal of WeakMap/WeakSet clear

2014-12-09 Thread Jason Orendorff
On Wed, Dec 3, 2014 at 3:54 PM, David Bruant bruan...@gmail.com wrote: There's something important that's implicit in this argument that I still don't have yet. If you were using literally any other data structure, any other object, passing a direct reference to it around to untrusted code

Re: Removal of WeakMap/WeakSet clear

2014-12-09 Thread Claude Pache
Le 5 déc. 2014 à 05:06, Alex Russell slightly...@google.com a écrit : I support Katelyn's suggestion to make clear() neuterable on an instance, perhaps with per-object configuration. It leaves the API intact while allowing those with security concerns to address them. What is the

Re: Removal of WeakMap/WeakSet clear

2014-12-09 Thread Jason Orendorff
On Thu, Dec 4, 2014 at 6:58 AM, David Bruant bruan...@gmail.com wrote: Sometimes you call functions you don't have written and pass arguments to them. WeakMaps are new, but APIs will have functions with WeakMaps as arguments. I don't see what's crazy. It'd be nice if I don't have to review all

Re: Removal of WeakMap/WeakSet clear

2014-12-09 Thread Allen Wirfs-Brock
On Dec 9, 2014, at 2:51 PM, Jason Orendorff wrote: On Mon, Dec 8, 2014 at 3:46 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Thu, Dec 4, 2014 at 9:46 PM, Katelyn Gadd k...@luminance.org wrote: I'm surprised to hear that JS runtimes don't necessarily have ways to 'hash' a given JS value

Re: Removal of WeakMap/WeakSet clear

2014-12-08 Thread Tab Atkins Jr.
On Thu, Dec 4, 2014 at 9:46 PM, Katelyn Gadd k...@luminance.org wrote: I'm surprised to hear that JS runtimes don't necessarily have ways to 'hash' a given JS value, but it makes sense. I can see how that is a great reason for 'get me a hash for this value' to never actually exist in the API,

Re: Removal of WeakMap/WeakSet clear

2014-12-08 Thread Brendan Eich
Tab Atkins Jr. wrote: JS has maps/sets that take objects natively, hiding any details about how a mutable object is tracked/stored as keys or values, so there's never been any need for such a thing. Explicitly exposing hash codes is leaking implementation details into the user-facing API. Not

Re: Removal of WeakMap/WeakSet clear

2014-12-08 Thread Benjamin Gruenbaum
Never been a need? Why if you want to use objects as keys and not use reference equality? We've been over this. Here's a ref https://mail.mozilla.org/pipermail/es-discuss/2014-February/036389.html On Dec 8, 2014, at 23:46, Tab Atkins Jr. jackalm...@gmail.com wrote: On Thu, Dec 4, 2014 at

Re: Removal of WeakMap/WeakSet clear

2014-12-04 Thread Andreas Rossberg
On 4 December 2014 at 00:54, David Bruant bruan...@gmail.com wrote: The way I see it, data structures are a tool to efficiently query data. They don't *have* to be arbitrarily mutable anytime for this purpose. It's a point of view problem, but in my opinion, mutability is the problem, not

Re: Removal of WeakMap/WeakSet clear

2014-12-04 Thread David Bruant
Le 04/12/2014 09:55, Andreas Rossberg a écrit : On 4 December 2014 at 00:54, David Bruant bruan...@gmail.com wrote: The way I see it, data structures are a tool to efficiently query data. They don't *have* to be arbitrarily mutable anytime for this purpose. It's a point of view problem, but in

Re: Removal of WeakMap/WeakSet clear

2014-12-04 Thread Katelyn Gadd
There are scenarios where both security and performance matter. I think this is more than self-evident at this point in the thread since examples of both have been provided repeatedly. 'can you demonstrate there are no such scenario' isn't really a necessary question because we already know the

Re: Removal of WeakMap/WeakSet clear

2014-12-04 Thread Andreas Rossberg
On 4 December 2014 at 13:58, David Bruant bruan...@gmail.com wrote: Le 04/12/2014 09:55, Andreas Rossberg a écrit : On 4 December 2014 at 00:54, David Bruant bruan...@gmail.com wrote: The way I see it, data structures are a tool to efficiently query data. They don't *have* to be arbitrarily

Re: Removal of WeakMap/WeakSet clear

2014-12-04 Thread C. Scott Ananian
On Thu, Dec 4, 2014 at 9:25 AM, Katelyn Gadd k...@luminance.org wrote: The only way I can imagine to solve this is to make really aggressive use of type information gathering and/or bailouts in the runtime to identify every type used as a weakmap key - at which point I suppose you would have

Re: Removal of WeakMap/WeakSet clear

2014-12-04 Thread Mark S. Miller
On Thu, Dec 4, 2014 at 6:25 AM, Katelyn Gadd k...@luminance.org wrote: [...] I should also note that while much of the above is speculative and based on intuition/experience, I *have* been shipping a use of WeakMap for performance-critical code for over a year now Hi Katelyn, could you say

Re: Removal of WeakMap/WeakSet clear

2014-12-04 Thread Katelyn Gadd
JSIL has a shim that emulates the 2D portion of the XNA game framework's graphics stack using HTML5 canvas (for compatibility). Many of the stack's features don't have direct equivalents in canvas, so I have to generate and cache various bits of data and graphics resources on-demand to implement

Re: Removal of WeakMap/WeakSet clear

2014-12-04 Thread Alex Russell
I support Katelyn's suggestion to make clear() neuterable on an instance, perhaps with per-object configuration. It leaves the API intact while allowing those with security concerns to address them. On 4 Dec 2014 20:01, Katelyn Gadd k...@luminance.org wrote: JSIL has a shim that emulates the 2D

Re: Removal of WeakMap/WeakSet clear

2014-12-04 Thread Steve Fink
On 12/04/2014 08:00 PM, Katelyn Gadd wrote: I do still use WeakMap in a few other places, for example to implement Object.GetHashCode. This is a case where the transposed representation is likely optimal - though in practice, I shouldn't need any sort of container here, if only the hashing

Re: Removal of WeakMap/WeakSet clear

2014-12-04 Thread Katelyn Gadd
.NET's hashing protocol is weird and arguably it's some awful baggage carried over from its Java influences. All instances, value or reference type, have GetHashCode. For a given type there is the 'default' implementation, or you can provide a specific one. This enables anything to be used as a

Re: Removal of WeakMap/WeakSet clear

2014-12-03 Thread Andreas Rossberg
On 2 December 2014 at 19:15, Mark S. Miller erig...@google.com wrote: In both minor or major collection both m and v are immediately reclaimed, because neither is strongly reachable at that point which shows the asymmetry, and that v8 is effectively optimizing for the wrong side of that

Re: Removal of WeakMap/WeakSet clear

2014-12-03 Thread Jason Orendorff
On Wed, Dec 3, 2014 at 8:35 AM, Andreas Rossberg rossb...@google.com wrote: (Back to the actual topic of this thread, you still owe me a reply regarding why .clear is bad for security. ;) ) I'd like to hear this too, just for education value. -j ___

Re: Removal of WeakMap/WeakSet clear

2014-12-03 Thread David Bruant
Le 03/12/2014 16:26, Jason Orendorff a écrit : On Wed, Dec 3, 2014 at 8:35 AM, Andreas Rossberg rossb...@google.com wrote: (Back to the actual topic of this thread, you still owe me a reply regarding why .clear is bad for security. ;) ) I'd like to hear this too, just for education value.

Re: Removal of WeakMap/WeakSet clear

2014-12-03 Thread Michał Wadas
The same argument can be used for removing array.length setter - some code can remove all entries from array important for your application. Maybe we should think about WM methods .clone and cloneInto(otherWeakMap). 3 gru 2014 18:04 David Bruant bruan...@gmail.com napisał(a): Le 03/12/2014

Re: Removal of WeakMap/WeakSet clear

2014-12-03 Thread Tab Atkins Jr.
On Wed, Dec 3, 2014 at 9:10 AM, Michał Wadas michalwa...@gmail.com wrote: The same argument can be used for removing array.length setter - some code can remove all entries from array important for your application. No it's not, because Arrays are open to editing in all ways. They don't have

Re: Removal of WeakMap/WeakSet clear

2014-12-03 Thread Chris Toshok
On Thu, Nov 27, 2014 at 10:40 AM, Allen Wirfs-Brock al...@wirfs-brock.com wrote: This is the end of my assumed inverted WC design and why I assert that a clear method is incompatible with it. Couldn't this be solved by adding a little state (a monotonically increasing 'generation' counter)

Re: Removal of WeakMap/WeakSet clear

2014-12-03 Thread Jason Orendorff
On Wed, Dec 3, 2014 at 9:04 AM, David Bruant bruan...@gmail.com wrote: A script which builds a weakmap may legitimately later assume the weakmap is filled. However, passing the weakmap to a mixed-trusted (malicious or buggy) script may result in the weakmap being cleared (and break the

Re: Removal of WeakMap/WeakSet clear

2014-12-03 Thread Rick Waldron
On Wed, Dec 3, 2014 at 1:10 PM, Jason Orendorff jason.orendo...@gmail.com wrote: On Wed, Dec 3, 2014 at 9:04 AM, David Bruant bruan...@gmail.com wrote: A script which builds a weakmap may legitimately later assume the weakmap is filled. However, passing the weakmap to a mixed-trusted

Re: Removal of WeakMap/WeakSet clear

2014-12-03 Thread Chris Toshok
A more compact implementation occurred to me during my commute: just have an internal slot in the WC that it uses when looking up values, instead of the WC reference itself. This has the downside of not being able to purge old slots on calls to has/get, but removes the possibility of overflow

Re: Removal of WeakMap/WeakSet clear

2014-12-03 Thread Allen Wirfs-Brock
On Dec 3, 2014, at 11:28 AM, Chris Toshok wrote: A more compact implementation occurred to me during my commute: just have an internal slot in the WC that it uses when looking up values, instead of the WC reference itself. This has the downside of not being able to purge old slots on

Re: Removal of WeakMap/WeakSet clear

2014-12-03 Thread Katelyn Gadd
Related to Andreas's points, if performance is a real concern here, the dramatically inferior memory locality of a transposed weak map is probably something that is at least worth thinking about. Cache locality matters a lot in modern software, especially if you have a large heap, and it will

Re: Removal of WeakMap/WeakSet clear

2014-12-03 Thread Katelyn Gadd
'It remains possible to create clearless weakmaps to pass around (by wrapping a weakmap, etc.), but it makes security (aka code robustness) an opt-in and not the default.' Seeing this sort of security-first argument when discussing a user-space data structure is really confusing to me. If the

Re: Removal of WeakMap/WeakSet clear

2014-12-03 Thread David Bruant
Le 03/12/2014 19:10, Jason Orendorff a écrit : On Wed, Dec 3, 2014 at 9:04 AM, David Bruantbruan...@gmail.com wrote: A script which builds a weakmap may legitimately later assume the weakmap is filled. However, passing the weakmap to a mixed-trusted (malicious or buggy) script may result in

Re: Removal of WeakMap/WeakSet clear

2014-12-02 Thread Andreas Rossberg
On 29 November 2014 at 22:30, Mark S. Miller erig...@google.com wrote: On Fri, Nov 28, 2014 at 4:21 AM, Andreas Rossberg rossb...@google.com wrote: [...] With a normal ephemeral weak map implementation, like the one in V8, the value in a (map,key,value) triple can be reclaimed _immediately_

Re: Removal of WeakMap/WeakSet clear

2014-12-02 Thread Allen Wirfs-Brock
On Dec 2, 2014, at 10:15 AM, Mark S. Miller wrote: Good. I think we're coming to a mutual understanding. By scavenge I mean exactly your minor collection. I think collecting typical garbage during minor collection, rather than promoting/tenuring it, is desperately important, and dominates

Re: Removal of WeakMap/WeakSet clear

2014-12-02 Thread Brendan Eich
Good point about multi-gen. Lars Hansen even researched oldest first collection: http://www.cesura17.net/~will/professional/research/presentations/gc/index.html Clearly, Ungar's model was a simplification, with trade-offs as expected. Allen Wirfs-Brock wrote: My main take-away from this

Re: Removal of WeakMap/WeakSet clear

2014-11-30 Thread Allen Wirfs-Brock
On Nov 28, 2014, at 4:21 AM, Andreas Rossberg wrote: On 27 November 2014 at 19:40, Allen Wirfs-Brock al...@wirfs-brock.com wrote: On Nov 27, 2014, at 1:17 AM, Andreas Rossberg wrote: The discussion here still makes various untested assumptions about the transposed representation. With

Re: Removal of WeakMap/WeakSet clear

2014-11-29 Thread Jason Orendorff
On Thu, Nov 27, 2014 at 8:58 AM, Andrea Giammarchi andrea.giammar...@gmail.com wrote: On Thu, Nov 27, 2014 at 2:44 PM, Andreas Rossberg rossb...@google.com wrote: Well, there is no functionally correct polyfill for WeakMaps that is actually weak, regardless of .clear. Please bear with me so

Re: Removal of WeakMap/WeakSet clear

2014-11-29 Thread Mark S. Miller
On Fri, Nov 28, 2014 at 4:21 AM, Andreas Rossberg rossb...@google.com wrote: [...] With a normal ephemeral weak map implementation, like the one in V8, the value in a (map,key,value) triple can be reclaimed _immediately_ (in the same cycle) when _either_ the map _or_ the key dies. Hi Andreas,

Re: Removal of WeakMap/WeakSet clear

2014-11-28 Thread Andreas Rossberg
On 27 November 2014 at 19:40, Allen Wirfs-Brock al...@wirfs-brock.com wrote: On Nov 27, 2014, at 1:17 AM, Andreas Rossberg wrote: The discussion here still makes various untested assumptions about the transposed representation. With respect to .clear in particular, it seems to assume that this

Re: Removal of WeakMap/WeakSet clear

2014-11-27 Thread Andreas Rossberg
The discussion here still makes various untested assumptions about the transposed representation. With respect to .clear in particular, it seems to assume that this representation does not normally need the ability to iterate (internally) over the keys of a weak map. AFAICT, that is incorrect.

Re: Removal of WeakMap/WeakSet clear

2014-11-27 Thread Andrea Giammarchi
Andreas I think Mark few times already gave for granted WeakMaps are based on transposed representations and it's also partially pointless from a polyfill point of view to not implement them as such but clear makes fast polyfills very pointless with `.clear()` in place ( e.g. [1] ) I've had

Re: Removal of WeakMap/WeakSet clear

2014-11-27 Thread Andreas Rossberg
On 27 November 2014 at 15:22, Andrea Giammarchi andrea.giammar...@gmail.com wrote: Andreas I think Mark few times already gave for granted WeakMaps are based on transposed representations and it's also partially pointless from a polyfill point of view to not implement them as such but clear

Re: Removal of WeakMap/WeakSet clear

2014-11-27 Thread Andrea Giammarchi
On Thu, Nov 27, 2014 at 2:44 PM, Andreas Rossberg rossb...@google.com wrote: Well, there is no functionally correct polyfill for WeakMaps that is actually weak, regardless of .clear. Please bear with me so I understand your point: if you have `o1` and `o2` and `o1.link = o2;`, wouldn't `o2`

Re: Removal of WeakMap/WeakSet clear

2014-11-27 Thread Andreas Rossberg
On 27 November 2014 at 15:58, Andrea Giammarchi andrea.giammar...@gmail.com wrote: On Thu, Nov 27, 2014 at 2:44 PM, Andreas Rossberg rossb...@google.com wrote: Well, there is no functionally correct polyfill for WeakMaps that is actually weak, regardless of .clear. Please bear with me so I

Re: Removal of WeakMap/WeakSet clear

2014-11-27 Thread Andrea Giammarchi
Right, the compromise then is usually to use a descriptor with `enumerable:false` that won't break the common web but agreed it's not that reliable or safe ... although developers are interested in the weak bit, and most of them most likely won't deal with new kind of objects such Proxies and/or

Re: Removal of WeakMap/WeakSet clear

2014-11-27 Thread Allen Wirfs-Brock
On Nov 27, 2014, at 1:17 AM, Andreas Rossberg wrote: The discussion here still makes various untested assumptions about the transposed representation. With respect to .clear in particular, it seems to assume that this representation does not normally need the ability to iterate (internally)

Re: Removal of WeakMap/WeakSet clear

2014-11-27 Thread Brendan Eich
(Nit-correcting along with comments and questions in-line, just trying to parse what you mean, not snarking -- except for the first comment :-D.) Allen Wirfs-Brock wrote: On Nov 27, 2014, at 1:17 AM, Andreas Rossberg wrote: The discussion here still makes various untested assumptions about

Re: Removal of WeakMap/WeakSet clear

2014-11-27 Thread Allen Wirfs-Brock
On Nov 27, 2014, at 11:49 AM, Brendan Eich wrote: (Nit-correcting along with comments and questions in-line, just trying to parse what you mean, not snarking -- except for the first comment :-D.) Allen Wirfs-Brock wrote: On Nov 27, 2014, at 1:17 AM, Andreas Rossberg wrote: The

Re: Removal of WeakMap/WeakSet clear

2014-11-27 Thread Brendan Eich
Allen Wirfs-Brock wrote: WeakMap get/set/has/delete work similarly by accessing keys' inverted map using M as the key. [A] This means M must be mapped to WC, via some internal member or (side-)side-table. M is an instance of a WC, eg, M is a WeakMap which is a kind of WC Oh I see --

Removal of WeakMap/WeakSet clear

2014-11-26 Thread Katelyn Gadd
Is there a detailed rationale for this somewhere? Making typical applications pay the cost here for a specific security scenario seems really bizarre to me. Clearing standard library data structures is an incredibly common operation. If you want to ensure that someone can't clear the map/set,

Re: Removal of WeakMap/WeakSet clear

2014-11-26 Thread Claude Pache
The root of the issue, is that WeakMaps are thought as a tool for two unrelated use cases. Quoting [1]: (...) WeakMaps were motivate[d] by two distinct use cases. Soft fields and object-keyed caches. Now, in short, for the soft fields use case, a `.clear()` method is unwanted, but for the

Re: Removal of WeakMap/WeakSet clear

2014-11-26 Thread Allen Wirfs-Brock
On Nov 26, 2014, at 10:22 AM, Claude Pache wrote: The root of the issue, is that WeakMaps are thought as a tool for two unrelated use cases. Quoting [1]: (...) WeakMaps were motivate[d] by two distinct use cases. Soft fields and object-keyed caches. Now, in short, for the soft

RE: Removal of WeakMap/WeakSet clear

2014-11-26 Thread Andrea Giammarchi
@mozilla.org Subject: Removal of WeakMap/WeakSet clear Is there a detailed rationale for this somewhere? Making typical applications pay the cost here for a specific security scenario seems really bizarre to me. Clearing standard library data structures is an incredibly common operation. If you want

Re: Removal of WeakMap/WeakSet clear

2014-11-26 Thread Mark S. Miller
On Wed, Nov 26, 2014 at 9:33 AM, Katelyn Gadd k...@luminance.org wrote: Is there a detailed rationale for this somewhere? It is a combination of three issues. 1. The security issue. 2. The implementation issue. 3. The ES process issue. The implementation issue is that the use cases for

Re: Removal of WeakMap/WeakSet clear

2014-11-26 Thread Mark S. Miller
On Wed, Nov 26, 2014 at 11:09 AM, Allen Wirfs-Brock al...@wirfs-brock.com wrote: On Nov 26, 2014, at 10:22 AM, Claude Pache wrote: The root of the issue, is that WeakMaps are thought as a tool for two unrelated use cases. Quoting [1]: (...) WeakMaps were motivate[d] by two distinct use

Re: Removal of WeakMap/WeakSet clear

2014-11-26 Thread Tab Atkins Jr.
On Wed, Nov 26, 2014 at 11:23 AM, Mark S. Miller erig...@google.com wrote: On Wed, Nov 26, 2014 at 11:09 AM, Allen Wirfs-Brock al...@wirfs-brock.com wrote: On Nov 26, 2014, at 10:22 AM, Claude Pache wrote: The root of the issue, is that WeakMaps are thought as a tool for two unrelated use

Re: Removal of WeakMap/WeakSet clear

2014-11-26 Thread Allen Wirfs-Brock
On Nov 26, 2014, at 11:25 AM, Tab Atkins Jr. wrote: On Wed, Nov 26, 2014 at 11:23 AM, Mark S. Miller erig...@google.com wrote: On Wed, Nov 26, 2014 at 11:09 AM, Allen Wirfs-Brock al...@wirfs-brock.com wrote: On Nov 26, 2014, at 10:22 AM, Claude Pache wrote: The root of the issue, is

Re: Removal of WeakMap/WeakSet clear

2014-11-26 Thread Katelyn Gadd
The detailed explanations are helpful for understanding this, at least when it comes to WeakMap. Thanks. I was not aware that clear() was a tenative addition - MDN (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WeakMap) lists it as available in all the major

Re: Removal of WeakMap/WeakSet clear

2014-11-26 Thread Brendan Eich
Katelyn Gadd wrote: However: Does this mean that if I implement clear myself by creating new WeakMaps, i'm inadvertently doing horrible things to the GC? Assume yes (only defensive assumption). What goes wrong? Sorry if I missed it: what is your .clear use case, in example code? /be

Re: Removal of WeakMap/WeakSet clear

2014-11-26 Thread Mark S. Miller
On Wed, Nov 26, 2014 at 2:49 PM, Katelyn Gadd k...@luminance.org wrote: The detailed explanations are helpful for understanding this, at least when it comes to WeakMap. Thanks. I was not aware that clear() was a tenative addition - MDN