Re: Death Before Confusion (was: [whatwg] Handling out of memory issues with getImageData/createImageData)

[Mark S. Miller]

On Sun, Sep 27, 2015 at 9:57 AM, Filip Pizlo  wrote:

> Hi Mark,
>
> It seems that most of the benefit for fail-faster behavior for VM errors
> is security.
>
> To what extent do you think the security problem could be addressed by VMs
> simply randomizing the point at which stack overflow or OOM happens?  I
> think this would be more desirable, since it requires no language changes.
>

It would help, but not enough. The defender would have some window of
memory budget within which it randomizes. The attacker could repeatedly
probe in order to get a statistical sense of the range and shape of that
window. Then, say, the attacker could repeatedly allocate until it got,
say, into the 80 percentile of that window without failing, and then call
the defender. If the defender then does a delicate operation that happens
to allocate more that the remaining 20 percentile of that window, then the
attacker has broken the defender's integrity in a way the attacker may be
able to exploit.

Nevertheless, it does help a lot. It would be a nice speed bump until a
real defense can be designed, agreed on, and put in place.

We also need real experiments to determine how hard these attacks actually
are to mount. And once successful, how hard they are to commodify, so that
other less skilled attackers can reuse these attacks. As always, if you do
this as a white hat, please engage in responsible disclosure for a
reasonable finite period before making a successful attack public. Thanks.




>
> More comments inline...
>
> On Sep 27, 2015, at 8:46 AM, Mark S. Miller  wrote:
>
> [-whatwg, +es-discuss]
> Reposting to es-discuss, as Anne's more general question is best seen as a
> JS issue rather than a browser specific one
>
>
>
> On Sun, Sep 27, 2015 at 8:30 AM, Mark S. Miller 
> wrote:
>
>> On Sat, Sep 26, 2015 at 7:34 AM, Anne van Kesteren 
>> wrote:
>>
>>> On Fri, Sep 25, 2015 at 4:48 PM, Justin Novosad 
>>> wrote:
>>> > Currently there is no spec'ed behavior for handling out-of memory
>>> issues
>>> > for the specific case of attempting to allocate a large buffer through
>>> > image data APIs.
>>>
>>> Actually, there is no specified behavior for out-of-memory behavior,
>>> period. This is a problem that starts with the ECMAScript standard and
>>> everything that builds upon it.
>>>
>>> I have seen Mark Miller discuss some of the issues surrounding this
>>> and perhaps even the necessity to eventually define it, but so far
>>> this has not happened. Not sure if the full story is documented
>>> somewhere. Mark?
>>>
>>>
>>> https://esdiscuss.org/topic/using-max-stack-limit-to-determine-current-js-engine-and-revision#content-7
>>> indicates there may be security issues with throwing out-of-memory
>>> exceptions.
>>
>>
>> Well, the full story is never documented ;). However, that post and the
>> links from there:
>>
>> http://www.eros-os.org/pipermail/e-lang/2007-January/011817.html
>> https://github.com/google/caja/issues/460
>>
>> are a good start. The security issue is serious and needs to be fixed. It
>> cannot practically be fixed by libraries without additional help by the
>> platform. The problem is that
>>
>> * In a language that implicitly allocates everywhere, like
>> JavaScript, Java, and many other oo languages, it is impossible to prevent
>> a code from causing OOM
>> * If OOM is thrown (see the first link for Java/Joe-E issues), and
>> the language has try/finally, it is impossible to prevent the OOM being
>> masked.
>> * In such languages, it is impossible to program defensively against
>> the pervasive possibility of OOM -- if execution simply resumes in that
>> context as if nothing bad happened.
>>
>> In Joe-E we took the painful step of outlawing the Java try/finally from
>> the Joe-E subset of Java for this reason. There was no other reason to
>> outlaw try/finally as there's nothing else inherently unsafe about it. We
>> really tried to find another solution but under our constraints -- no
>> rewriting of the Java nor change to the JVM -- we could not.
>>
>> By preventing Joe-E code from catching VirtualMachineErrors and from
>> doing a try/finally, the Joe-E code was preemptively terminated immediately
>> on occurrence of a VirtualMachineError. Only the spawner of the Joe-E
>> computation could react to this termination of the computation it spawned.
>>
>> This mirrors one of the many thing that Erlang gets right. When a program
>> is confused, that program is the last one you want to ask to recover from
>> the confusion, since it is already impaired by its own confusion. If you
>> don't know what is still true, you are unlikely to engage in repair actions
>> correctly. Better to preemptively terminate some large unit containing the
>> confusion and recover by
>> * restarting from earlier known good state, or
>> * if this is not yet feasible, propagating the termination to a yet
>> larger 

Re: Weak References

[Herby Vojčík]

A naive question: don't you get working weakref by using a WeakMap and 
getting and setting using the same key? A naive point of view suggests 
that. What am I not seeing here?


Herby

Isiah Meadows wrote:

I'm resurrecting this [1] because I have found a use case where I needed
a weak reference. Is there any chance this could get into the language?

Here's my particular use case: making a stream with a rotating
destination that itself acts as a stream (snippet of that code, or what
I'd like it to be).

```js
function rotateStream(interval, format) {
 // Without the weak reference here, there's a memory leak
 const ref = new WeakReference({})
 setStream(ref.get(), format)
 setInterval(function () {
 if (ref.exists()) setStream(ref.get(), format)
 else clearInterval(this)
 }, interval)
 return ref.get()
}
```

This basically rotates an active stream, with the weak reference
pointing to the said stream. The alternative requires explicit marking
(in this case, with a `close` method):

```js
function leakyRotateStream(interval, format) {
 let stream = {close() {
 clearInterval(timer)
 timer = stream = null
 }}
 const timer = setInterval(function () {
 setStream(stream, format)
 }, interval)
 setStream(stream, format)
 return stream
}
```

In this use case, weak references would simplify the implementation and
reduce memory costs. And I would rather take advantage of the GC
machinery than explicit memory management (as explicit as C), as it
would be easier to collect when that's the only thing referenced.

(The setInterval callback is gc'd with all its references when it's
cleared from within.)

Thankfully, I'm using Node.js, so `weak` [2] is an option (the only
option, really). But I would definitely appreciate if it was available
in JS proper, across engines. V8, SpiderMonkey, and JSC all have weak
references to JS objects as part of their public API, and I would be
surprised if the implementation work would be anything significant.

[1]: https://esdiscuss.org/topic/what-is-the-status-of-weak-references
[2]: https://github.com/TooTallNate/node-weak

--
Isiah Meadows

___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss

___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss

Re: Death Before Confusion (was: [whatwg] Handling out of memory issues with getImageData/createImageData)

[Filip Pizlo]

Hi Mark,

It seems that most of the benefit for fail-faster behavior for VM errors is 
security. 

To what extent do you think the security problem could be addressed by VMs 
simply randomizing the point at which stack overflow or OOM happens?  I think 
this would be more desirable, since it requires no language changes. 

More comments inline...

> On Sep 27, 2015, at 8:46 AM, Mark S. Miller  wrote:
> 
> [-whatwg, +es-discuss] 
> Reposting to es-discuss, as Anne's more general question is best seen as a JS 
> issue rather than a browser specific one
> 
> 
> 
>> On Sun, Sep 27, 2015 at 8:30 AM, Mark S. Miller  wrote:
>>> On Sat, Sep 26, 2015 at 7:34 AM, Anne van Kesteren  wrote:
>>> On Fri, Sep 25, 2015 at 4:48 PM, Justin Novosad  wrote:
>>> > Currently there is no spec'ed behavior for handling out-of memory issues
>>> > for the specific case of attempting to allocate a large buffer through
>>> > image data APIs.
>>> 
>>> Actually, there is no specified behavior for out-of-memory behavior,
>>> period. This is a problem that starts with the ECMAScript standard and
>>> everything that builds upon it.
>>> 
>>> I have seen Mark Miller discuss some of the issues surrounding this
>>> and perhaps even the necessity to eventually define it, but so far
>>> this has not happened. Not sure if the full story is documented
>>> somewhere. Mark?
>>> 
>>> https://esdiscuss.org/topic/using-max-stack-limit-to-determine-current-js-engine-and-revision#content-7
>>> indicates there may be security issues with throwing out-of-memory
>>> exceptions.
>> 
>> Well, the full story is never documented ;). However, that post and the 
>> links from there:
>> 
>> http://www.eros-os.org/pipermail/e-lang/2007-January/011817.html
>> https://github.com/google/caja/issues/460
>> 
>> are a good start. The security issue is serious and needs to be fixed. It 
>> cannot practically be fixed by libraries without additional help by the 
>> platform. The problem is that
>> 
>> * In a language that implicitly allocates everywhere, like JavaScript, 
>> Java, and many other oo languages, it is impossible to prevent a code from 
>> causing OOM
>> * If OOM is thrown (see the first link for Java/Joe-E issues), and the 
>> language has try/finally, it is impossible to prevent the OOM being masked.
>> * In such languages, it is impossible to program defensively against the 
>> pervasive possibility of OOM -- if execution simply resumes in that context 
>> as if nothing bad happened.
>> 
>> In Joe-E we took the painful step of outlawing the Java try/finally from the 
>> Joe-E subset of Java for this reason. There was no other reason to outlaw 
>> try/finally as there's nothing else inherently unsafe about it. We really 
>> tried to find another solution but under our constraints -- no rewriting of 
>> the Java nor change to the JVM -- we could not.
>> 
>> By preventing Joe-E code from catching VirtualMachineErrors and from doing a 
>> try/finally, the Joe-E code was preemptively terminated immediately on 
>> occurrence of a VirtualMachineError. Only the spawner of the Joe-E 
>> computation could react to this termination of the computation it spawned. 
>> 
>> This mirrors one of the many thing that Erlang gets right. When a program is 
>> confused, that program is the last one you want to ask to recover from the 
>> confusion, since it is already impaired by its own confusion. If you don't 
>> know what is still true, you are unlikely to engage in repair actions 
>> correctly. Better to preemptively terminate some large unit containing the 
>> confusion and recover by 
>> * restarting from earlier known good state, or
>> * if this is not yet feasible, propagating the termination to a yet 
>> larger granularity of computation.
>> 
>> This is the "fail stop" philosophy of "Death Before Confusion". The 
>> contrasting philosophy appropriate for some computation is "best efforts". 
>> Some JavaScript code is best served by one and some by the other. Security 
>> enforcing code must maintain its own integrity at the price of termination 
>> (and restart from some coarser grain). Web pages using JavaScript only to 
>> spice up the user experience are often best served by best efforts. Erlang 
>> itself is an interesting case study, as its original motivating problem -- 
>> telephone switches -- places a higher priority on uptime than on integrity. 
>> Nevertheless, both Erlang and the Tandem non-stop architecture found that 
>> uptime in the large is best served by fail-stop in the small combined with 
>> coarser-grain recovery logic.
>> 
>> Because JavaScript comes from such a long legacy of de facto best efforts 
>> architecture, I think a direct du jure shift to fail-stop is unlikely. 
>> Instead, what we need is a trap-handling mechanism (Erlang "supervisor". 
>> KeyKOS "keeper"), where different policies can be expressed by user-defined 
>> trap handlers. When 

Death Before Confusion (was: [whatwg] Handling out of memory issues with getImageData/createImageData)

[Mark S. Miller]

[-whatwg, +es-discuss]
Reposting to es-discuss, as Anne's more general question is best seen as a
JS issue rather than a browser specific one



On Sun, Sep 27, 2015 at 8:30 AM, Mark S. Miller  wrote:

> On Sat, Sep 26, 2015 at 7:34 AM, Anne van Kesteren 
> wrote:
>
>> On Fri, Sep 25, 2015 at 4:48 PM, Justin Novosad  wrote:
>> > Currently there is no spec'ed behavior for handling out-of memory issues
>> > for the specific case of attempting to allocate a large buffer through
>> > image data APIs.
>>
>> Actually, there is no specified behavior for out-of-memory behavior,
>> period. This is a problem that starts with the ECMAScript standard and
>> everything that builds upon it.
>>
>> I have seen Mark Miller discuss some of the issues surrounding this
>> and perhaps even the necessity to eventually define it, but so far
>> this has not happened. Not sure if the full story is documented
>> somewhere. Mark?
>>
>>
>> https://esdiscuss.org/topic/using-max-stack-limit-to-determine-current-js-engine-and-revision#content-7
>> indicates there may be security issues with throwing out-of-memory
>> exceptions.
>
>
> Well, the full story is never documented ;). However, that post and the
> links from there:
>
> http://www.eros-os.org/pipermail/e-lang/2007-January/011817.html
> https://github.com/google/caja/issues/460
>
> are a good start. The security issue is serious and needs to be fixed. It
> cannot practically be fixed by libraries without additional help by the
> platform. The problem is that
>
> * In a language that implicitly allocates everywhere, like JavaScript,
> Java, and many other oo languages, it is impossible to prevent a code from
> causing OOM
> * If OOM is thrown (see the first link for Java/Joe-E issues), and the
> language has try/finally, it is impossible to prevent the OOM being masked.
> * In such languages, it is impossible to program defensively against
> the pervasive possibility of OOM -- if execution simply resumes in that
> context as if nothing bad happened.
>
> In Joe-E we took the painful step of outlawing the Java try/finally from
> the Joe-E subset of Java for this reason. There was no other reason to
> outlaw try/finally as there's nothing else inherently unsafe about it. We
> really tried to find another solution but under our constraints -- no
> rewriting of the Java nor change to the JVM -- we could not.
>
> By preventing Joe-E code from catching VirtualMachineErrors and from doing
> a try/finally, the Joe-E code was preemptively terminated immediately on
> occurrence of a VirtualMachineError. Only the spawner of the Joe-E
> computation could react to this termination of the computation it spawned.
>
> This mirrors one of the many thing that Erlang gets right. When a program
> is confused, that program is the last one you want to ask to recover from
> the confusion, since it is already impaired by its own confusion. If you
> don't know what is still true, you are unlikely to engage in repair actions
> correctly. Better to preemptively terminate some large unit containing the
> confusion and recover by
> * restarting from earlier known good state, or
> * if this is not yet feasible, propagating the termination to a yet
> larger granularity of computation.
>
> This is the "fail stop" philosophy of "Death Before Confusion". The
> contrasting philosophy appropriate for some computation is "best efforts".
> Some JavaScript code is best served by one and some by the other. Security
> enforcing code must maintain its own integrity at the price of termination
> (and restart from some coarser grain). Web pages using JavaScript only to
> spice up the user experience are often best served by best efforts. Erlang
> itself is an interesting case study, as its original motivating problem --
> telephone switches -- places a higher priority on uptime than on integrity.
> Nevertheless, both Erlang and the Tandem non-stop architecture found that
> uptime in the large is best served by fail-stop in the small combined with
> coarser-grain recovery logic.
>
> Because JavaScript comes from such a long legacy of de facto best efforts
> architecture, I think a direct du jure shift to fail-stop is unlikely.
> Instead, what we need is a trap-handling mechanism (Erlang "supervisor".
> KeyKOS "keeper"), where different policies can be expressed by user-defined
> trap handlers. When multiple policies co-exist, the platform obeys the
> more severe policies. For concreteness, I'll make here a first sketch:
>
> On OOM, the platform first scans the stack to enumerate all realms
> represented by in-progress stack frames as of that moment. (In progress
> meaning that the stack frame still would have been there even if that
> platform had implemented proper-tail-call.) It gathers the trap handlers
> associated with each of those realms. Each trap handler is a pair of a
> string and an optional function.
>
> The string indicates the choice of 

Re: Weak References

[Jordan Harband]

`WeakMap` holds the *key* weakly, not the value - it holds the value
strongly. If you have the key such that you can use it to get at the value,
you'll always have the value.

`WeakRef` is a container around a weakly held *value*, which I don't
believe is possible to fully polyfill, even with `WeakMap`/`WeakSet`
(because they are not enumerable).

On Sun, Sep 27, 2015 at 9:09 AM, Herby Vojčík  wrote:

> A naive question: don't you get working weakref by using a WeakMap and
> getting and setting using the same key? A naive point of view suggests
> that. What am I not seeing here?
>
> Herby
>
> Isiah Meadows wrote:
>
>> I'm resurrecting this [1] because I have found a use case where I needed
>> a weak reference. Is there any chance this could get into the language?
>>
>> Here's my particular use case: making a stream with a rotating
>> destination that itself acts as a stream (snippet of that code, or what
>> I'd like it to be).
>>
>> ```js
>> function rotateStream(interval, format) {
>>  // Without the weak reference here, there's a memory leak
>>  const ref = new WeakReference({})
>>  setStream(ref.get(), format)
>>  setInterval(function () {
>>  if (ref.exists()) setStream(ref.get(), format)
>>  else clearInterval(this)
>>  }, interval)
>>  return ref.get()
>> }
>> ```
>>
>> This basically rotates an active stream, with the weak reference
>> pointing to the said stream. The alternative requires explicit marking
>> (in this case, with a `close` method):
>>
>> ```js
>> function leakyRotateStream(interval, format) {
>>  let stream = {close() {
>>  clearInterval(timer)
>>  timer = stream = null
>>  }}
>>  const timer = setInterval(function () {
>>  setStream(stream, format)
>>  }, interval)
>>  setStream(stream, format)
>>  return stream
>> }
>> ```
>>
>> In this use case, weak references would simplify the implementation and
>> reduce memory costs. And I would rather take advantage of the GC
>> machinery than explicit memory management (as explicit as C), as it
>> would be easier to collect when that's the only thing referenced.
>>
>> (The setInterval callback is gc'd with all its references when it's
>> cleared from within.)
>>
>> Thankfully, I'm using Node.js, so `weak` [2] is an option (the only
>> option, really). But I would definitely appreciate if it was available
>> in JS proper, across engines. V8, SpiderMonkey, and JSC all have weak
>> references to JS objects as part of their public API, and I would be
>> surprised if the implementation work would be anything significant.
>>
>> [1]: https://esdiscuss.org/topic/what-is-the-status-of-weak-references
>> [2]: https://github.com/TooTallNate/node-weak
>>
>> --
>> Isiah Meadows
>>
>> ___
>> es-discuss mailing list
>> es-discuss@mozilla.org
>> https://mail.mozilla.org/listinfo/es-discuss
>>
> ___
> es-discuss mailing list
> es-discuss@mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss
>
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss

Re: Exponentiation operator precedence

[Kevin Reid]

On Thu, Sep 24, 2015 at 8:14 AM, Mark S. Miller  wrote:

> I like #4. Normally in a situation like this I would still argue for #1.
> #4 is a complicated special case that breaks the normal pattern of operator
> precedence elsewhere in the language. The need for ** is not great enough
> to justify introducing a new special case for users to learn.
>
> However, in this case, #4 is only technically complicated -- for those
> writing or reading spec docs like us. For normal users, the only complexity
> is a rarely encountered surprising static error. With a decent (and easy to
> generate) error message, these users will immediately know what they need
> to do to repair their program.
>
> Significant programs are read much more than they are written. Both #2 and
> #3 will lead many readers to misread programs. For programs that are not
> rejected, #4 is no more confusing than #1. Altogether, for readers, #4 is
> better than #1 because ** is more readable than Pow.
>

MarkM, I'm surprised you didn't also mention that there is precedent for #4
from your own E.

The E language chose to place math operations as methods on numbers, rather
than on any static "Math" object, and does not have an exponentiation
operator. In order to avoid precedence surprises of the category we're
discussing, E statically rejects the combination of a unary prefix
(negation) and unary postfix (method call) operator.

-(2).pow(2)   # "ought to be" -4, is a syntax error
-(2).max(2)   # "ought to be" 2, is a syntax error

(The parentheses around the number are not actually required in E, but I
have included them for the sake of comparison to JS despite the lexical
rejection of "1.foo" in JS.)

JavaScript already syntactically accepts the above programs (parsing "-" as
lower precedence than ".foo()"), but #4 is in the same spirit of rejecting
cases where there are conflicting or unclear precedents for operator
precedence.
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss

Re: Re: Death Before Confusion (was: [whatwg] Handling out of memory issues with getImageData/createImageData)

[Ron Waldon]

Android has an older onLowMemory() callback and a newer onTrimMemory()
callback:
-
http://developer.android.com/reference/android/content/ComponentCallbacks.html#onLowMemory
()
-
http://developer.android.com/reference/android/content/ComponentCallbacks2.html#onTrimMemory(int
)

iOS has something similar as well.

Is making these available in ECMAScript proper or an annex a potential
solution to this class of problem?
___
es-discuss mailing list
es-discuss@mozilla.org
https://mail.mozilla.org/listinfo/es-discuss