Re: [whatwg] Handling out of memory issues with getImageData/createImageData

2015-09-28 Thread Justin Novosad 
On Mon, Sep 28, 2015 at 5:40 AM, Anne van Kesteren  wrote:

> On Sun, Sep 27, 2015 at 6:14 PM, Allen Wirfs-Brock
>  wrote:
> > Actually, that's not completely correct.  Within ES2015, the only way
> > explicitly allocate a large, dense area of memory is by creating a large
> > ArrayBuffer instance.  All attempts to create such instances eventually
> > perform the actions specified by CreateDataBlock [1].  CreateDataBlock
> > explicitly says that a RangeError exception is thrown if it is
> impossible to
> > allocated the requested memory.
> >
> > I would expect and future ES features that exposed similar allocation
> > capabilities to follow that same pattern.
> >
> > [1]: http://ecma-international.org/ecma-262/6.0/#sec-createbytedatablock
>
> Thank you Allen. It would then make the most sense to rethrow that
> exception for these  methods (and ImageData constructor). I
> filed https://github.com/whatwg/html/issues/197 to track this. If
> anyone is interested in working on this let me know. It should be a
> relatively straightforward refactoring of the existing descriptions of
> these features.
>
>
Okay. That settles it for me, future sophistication not withstanding.

To give more context on the Canvas/ArrayBuffer specific cases: These APIs
allow apps to request arbitrarily large contiguous memory buffers, so they
are frequent sites of allocation failure errors. Often when an allocation
failure happens in these API's the system is not critically low on memory,
and the rest of the application/browser may very well be able to continue
to function normally. For a long time in Blink, we've been mitigating OOM
crashes by allowing canvases to silently fail when their backing stores
fail to allocate. These failures have been implemented in a way that is
transparent to the javascript code. The only side effect is that the canvas
contents remain blank (I know, it's a big side effect). This strategy has
avoided the confusion issue that Mark explained. The only observable effect
form the app's perspective are when doing canvas readbacks, the results
come back blank.  Whether or not this is an acceptable user experience
depends on the nature of the application and whether or not the canvas is
central to the app, but it is almost always a better user experience than a
crash.

The additional problem we have with the ImageData creation APIs, is that
failing silently is a lot trickier because the APIs must return a valid
ImageData object of the correct size. Failure to do so would not be spec
compliant, could result in a confused state, and would likely result in
some hard to foresee exception being thrown later down the road. What Rik
was suggesting about returning a fake ImageData object would be consistent
with the silent failure/best effort approach we've been using so far, but
it has performance implications that we can't ignore: We would have to add
a conditional branch (or a function pointer indirection) to the
UInt8ClampedArray element accessor to differentiate the fake vs real
behaviors. This is the stuff of inner loops for image processing algorithms.

Throwing a RangeError exception in these cases does seem like the lesser
evil, not to mention that it is what the spec now calls for (thanks Allen).

Re: [whatwg] Handling out of memory issues with getImageData/createImageData

2015-09-28 Thread Anne van Kesteren 
On Sun, Sep 27, 2015 at 6:14 PM, Allen Wirfs-Brock
 wrote:
> Actually, that's not completely correct.  Within ES2015, the only way
> explicitly allocate a large, dense area of memory is by creating a large
> ArrayBuffer instance.  All attempts to create such instances eventually
> perform the actions specified by CreateDataBlock [1].  CreateDataBlock
> explicitly says that a RangeError exception is thrown if it is impossible to
> allocated the requested memory.
>
> I would expect and future ES features that exposed similar allocation
> capabilities to follow that same pattern.
>
> [1]: http://ecma-international.org/ecma-262/6.0/#sec-createbytedatablock

Thank you Allen. It would then make the most sense to rethrow that
exception for these  methods (and ImageData constructor). I
filed https://github.com/whatwg/html/issues/197 to track this. If
anyone is interested in working on this let me know. It should be a
relatively straightforward refactoring of the existing descriptions of
these features.


-- 
https://annevankesteren.nl/

Re: [whatwg] Handling out of memory issues with getImageData/createImageData

2015-09-27 Thread Mark S. Miller 
On Sat, Sep 26, 2015 at 7:34 AM, Anne van Kesteren  wrote:

> On Fri, Sep 25, 2015 at 4:48 PM, Justin Novosad  wrote:
> > Currently there is no spec'ed behavior for handling out-of memory issues
> > for the specific case of attempting to allocate a large buffer through
> > image data APIs.
>
> Actually, there is no specified behavior for out-of-memory behavior,
> period. This is a problem that starts with the ECMAScript standard and
> everything that builds upon it.
>
> I have seen Mark Miller discuss some of the issues surrounding this
> and perhaps even the necessity to eventually define it, but so far
> this has not happened. Not sure if the full story is documented
> somewhere. Mark?
>
>
> https://esdiscuss.org/topic/using-max-stack-limit-to-determine-current-js-engine-and-revision#content-7
> indicates there may be security issues with throwing out-of-memory
> exceptions.


Well, the full story is never documented ;). However, that post and the
links from there:

http://www.eros-os.org/pipermail/e-lang/2007-January/011817.html
https://github.com/google/caja/issues/460

are a good start. The security issue is serious and needs to be fixed. It
cannot practically be fixed by libraries without additional help by the
platform. The problem is that

* In a language that implicitly allocates everywhere, like JavaScript,
Java, and many other oo languages, it is impossible to prevent a code from
causing OOM
* If OOM is thrown (see the first link for Java/Joe-E issues), and the
language has try/finally, it is impossible to prevent the OOM being masked.
* In such languages, it is impossible to program defensively against
the pervasive possibility of OOM -- if execution simply resumes in that
context as if nothing bad happened.

In Joe-E we took the painful step of outlawing the Java try/finally from
the Joe-E subset of Java for this reason. There was no other reason to
outlaw try/finally as there's nothing else inherently unsafe about it. We
really tried to find another solution but under our constraints -- no
rewriting of the Java nor change to the JVM -- we could not.

By preventing Joe-E code from catching VirtualMachineErrors and from doing
a try/finally, the Joe-E code was preemptively terminated immediately on
occurrence of a VirtualMachineError. Only the spawner of the Joe-E
computation could react to this termination of the computation it spawned.

This mirrors one of the many thing that Erlang gets right. When a program
is confused, that program is the last one you want to ask to recover from
the confusion, since it is already impaired by its own confusion. If you
don't know what is still true, you are unlikely to engage in repair actions
correctly. Better to preemptively terminate some large unit containing the
confusion and recover by
* restarting from earlier known good state, or
* if this is not yet feasible, propagating the termination to a yet
larger granularity of computation.

This is the "fail stop" philosophy of "Death Before Confusion". The
contrasting philosophy appropriate for some computation is "best efforts".
Some JavaScript code is best served by one and some by the other. Security
enforcing code must maintain its own integrity at the price of termination
(and restart from some coarser grain). Web pages using JavaScript only to
spice up the user experience are often best served by best efforts. Erlang
itself is an interesting case study, as its original motivating problem --
telephone switches -- places a higher priority on uptime than on integrity.
Nevertheless, both Erlang and the Tandem non-stop architecture found that
uptime in the large is best served by fail-stop in the small combined with
coarser-grain recovery logic.

Because JavaScript comes from such a long legacy of de facto best efforts
architecture, I think a direct du jure shift to fail-stop is unlikely.
Instead, what we need is a trap-handling mechanism (Erlang "supervisor".
KeyKOS "keeper"), where different policies can be expressed by user-defined
trap handlers. When multiple policies co-exist,  For concreteness, I'll
make here a first sketch:

On OOM, the platform first scans the stack to enumerate all realms
represented by in-progress stack frames as of that moment. (In progress
meaning that the stack frame still would have been there even if that
platform had implemented proper-tail-call.) It gathers the trap handlers
associated with each of those realms. Each trap handler is a pair of a
string and an optional function.

The string indicates the choice of trap handling strategy, where these
strategies are ordered by severity. Among the gathered strategies, the most
severe win and the rest are discarded. From least to most severe, they are

"THROW"
"ABORT_JOB"
"REFRESH"
"ABORT_EVENT_LOOP"

Except for "THROW", all the rest cause the current turn/job to first be
preemptively terminated without running catch or finally blocks. If during
any one trap handling

Re: [whatwg] Handling out of memory issues with getImageData/createImageData

2015-09-27 Thread Mark S. Miller 
On Sat, Sep 26, 2015 at 3:33 PM, Niels Keurentjes <
niels.keurent...@omines.com> wrote:

> >
> https://esdiscuss.org/topic/using-max-stack-limit-to-determine-current-js-engine-and-revision#content-7
> > indicates there may be security issues with throwing out-of-memory
> exceptions.
>
> That's hardly worth considering. The technique described there for
> fingerprinting


Please read beyond the title. The important issue is not fingerprinting or
information leakage. It is loss of integrity.



> is interesting from a theorist's perspective, but exposing no data that
> cannot already be more reliably extracted from navigator.userAgent with a
> simple regex.
>
> An out-of-memory in a sandbox is just exposing information about the
> sandbox, and worth nothing therefore if the sandbox version itself isn’t
> already compromised, at which point the user is generally screwed anyway if
> he didn't patch in time. Being able to detect a vulnerability is not a
> prerequisite for exploiting it.
>
> Niels
>
> -Original Message-
> From: whatwg [mailto:whatwg-boun...@lists.whatwg.org] On Behalf Of Anne
> van Kesteren
> Sent: zaterdag 26 september 2015 16:35
> To: Justin Novosad <ju...@google.com>
> Cc: WHAT Working Group <wha...@whatwg.org>; Mark Miller <erig...@gmail.com
> >
> Subject: Re: [whatwg] Handling out of memory issues with
> getImageData/createImageData
>
> On Fri, Sep 25, 2015 at 4:48 PM, Justin Novosad <ju...@google.com> wrote:
> > Currently there is no spec'ed behavior for handling out-of memory issues
> > for the specific case of attempting to allocate a large buffer through
> > image data APIs.
>
> Actually, there is no specified behavior for out-of-memory behavior,
> period. This is a problem that starts with the ECMAScript standard and
> everything that builds upon it.
>
> I have seen Mark Miller discuss some of the issues surrounding this
> and perhaps even the necessity to eventually define it, but so far
> this has not happened. Not sure if the full story is documented
> somewhere. Mark?
>
>
> https://esdiscuss.org/topic/using-max-stack-limit-to-determine-current-js-engine-and-revision#content-7
> indicates there may be security issues with throwing out-of-memory
> exceptions.
>
>
> --
> https://annevankesteren.nl/
>



-- 
Cheers,
--MarkM

Re: [whatwg] Handling out of memory issues with getImageData/createImageData

2015-09-27 Thread Mark S. Miller 
Since my post is about the more general OOM issue, I have shifted the
discussion to es-discuss
https://mail.mozilla.org/pipermail/es-discuss/2015-September/044267.html
Please continue discussion of the non-browser-specific issue there.


On Sun, Sep 27, 2015 at 8:33 AM, Mark S. Miller  wrote:

> I should make it clear that my post is not concerned about OOM for image
> data -- the original subject of this thread -- but rather about the more
> general OOM question that Anne asks about.
>
>
> On Sat, Sep 26, 2015 at 9:15 PM, Rik Cabanier  wrote:
>
>> On Fri, Sep 25, 2015 at 7:51 AM, Boris Zbarsky  wrote:
>>
>> > On 9/25/15 10:48 AM, Justin Novosad wrote:
>> >
>> >> I am sharing this here in case there would be interest in standardizing
>> >> this behavior.
>> >>
>> >
>> > I personally think it's a good idea (and throwing an exception is how
>> > Gecko handles, or at least aims to handle, this situation).
>>
>>
>> In the past, we discussed that error conditions such as this shouldn't
>> throw exceptions. Most of the time, this type of error is just temporal
>> and
>> is resolved in the next frame. Rare exceptions are almost never caught by
>> the author so the application crashes.
>>
>> Maybe for out of memory conditions, we could return a fake imagedata
>> object
>> with nothing but transparent white. In addition an 'isValid' property
>> could
>> signal if you have a real imagedata object.
>>
>
>
>
> --
> Cheers,
> --MarkM
>



-- 
Cheers,
--MarkM

Re: [whatwg] Handling out of memory issues with getImageData/createImageData

2015-09-27 Thread Allen Wirfs-Brock 

On Sep 26, 2015, at 7:34 AM, Anne van Kesteren wrote:

> On Fri, Sep 25, 2015 at 4:48 PM, Justin Novosad  wrote:
>> Currently there is no spec'ed behavior for handling out-of memory issues
>> for the specific case of attempting to allocate a large buffer through
>> image data APIs.
> 
> Actually, there is no specified behavior for out-of-memory behavior,
> period. This is a problem that starts with the ECMAScript standard and
> everything that builds upon it.

Actually, that's not completely correct.  Within ES2015, the only way 
explicitly allocate a large, dense area of memory is by creating a large 
ArrayBuffer instance.  All attempts to create such instances eventually perform 
the actions specified by CreateDataBlock [1].  CreateDataBlock explicitly says 
that a RangeError exception is thrown if it is impossible to allocated the 
requested memory. 

I would expect and future ES features that exposed similar allocation 
capabilities to follow that same pattern.

Allen

[1]: http://ecma-international.org/ecma-262/6.0/#sec-createbytedatablock 









> 
> I have seen Mark Miller discuss some of the issues surrounding this
> and perhaps even the necessity to eventually define it, but so far
> this has not happened. Not sure if the full story is documented
> somewhere. Mark?
> 
> https://esdiscuss.org/topic/using-max-stack-limit-to-determine-current-js-engine-and-revision#content-7
> indicates there may be security issues with throwing out-of-memory
> exceptions.
> 
> 
> -- 
> https://annevankesteren.nl/
>

Re: [whatwg] Handling out of memory issues with getImageData/createImageData

2015-09-27 Thread Mark S. Miller 
I should make it clear that my post is not concerned about OOM for image
data -- the original subject of this thread -- but rather about the more
general OOM question that Anne asks about.


On Sat, Sep 26, 2015 at 9:15 PM, Rik Cabanier  wrote:

> On Fri, Sep 25, 2015 at 7:51 AM, Boris Zbarsky  wrote:
>
> > On 9/25/15 10:48 AM, Justin Novosad wrote:
> >
> >> I am sharing this here in case there would be interest in standardizing
> >> this behavior.
> >>
> >
> > I personally think it's a good idea (and throwing an exception is how
> > Gecko handles, or at least aims to handle, this situation).
>
>
> In the past, we discussed that error conditions such as this shouldn't
> throw exceptions. Most of the time, this type of error is just temporal and
> is resolved in the next frame. Rare exceptions are almost never caught by
> the author so the application crashes.
>
> Maybe for out of memory conditions, we could return a fake imagedata object
> with nothing but transparent white. In addition an 'isValid' property could
> signal if you have a real imagedata object.
>



-- 
Cheers,
--MarkM

Re: [whatwg] Handling out of memory issues with getImageData/createImageData

2015-09-27 Thread Mark S. Miller 
On Sun, Sep 27, 2015 at 8:30 AM, Mark S. Miller  wrote:

> On Sat, Sep 26, 2015 at 7:34 AM, Anne van Kesteren 
> wrote:
>
>> On Fri, Sep 25, 2015 at 4:48 PM, Justin Novosad  wrote:
>> > Currently there is no spec'ed behavior for handling out-of memory issues
>> > for the specific case of attempting to allocate a large buffer through
>> > image data APIs.
>>
>> Actually, there is no specified behavior for out-of-memory behavior,
>> period. This is a problem that starts with the ECMAScript standard and
>> everything that builds upon it.
>>
>> I have seen Mark Miller discuss some of the issues surrounding this
>> and perhaps even the necessity to eventually define it, but so far
>> this has not happened. Not sure if the full story is documented
>> somewhere. Mark?
>>
>>
>> https://esdiscuss.org/topic/using-max-stack-limit-to-determine-current-js-engine-and-revision#content-7
>> indicates there may be security issues with throwing out-of-memory
>> exceptions.
>
>
> Well, the full story is never documented ;). However, that post and the
> links from there:
>
> http://www.eros-os.org/pipermail/e-lang/2007-January/011817.html
> https://github.com/google/caja/issues/460
>
> are a good start. The security issue is serious and needs to be fixed. It
> cannot practically be fixed by libraries without additional help by the
> platform. The problem is that
>
> * In a language that implicitly allocates everywhere, like JavaScript,
> Java, and many other oo languages, it is impossible to prevent a code from
> causing OOM
> * If OOM is thrown (see the first link for Java/Joe-E issues), and the
> language has try/finally, it is impossible to prevent the OOM being masked.
> * In such languages, it is impossible to program defensively against
> the pervasive possibility of OOM -- if execution simply resumes in that
> context as if nothing bad happened.
>
> In Joe-E we took the painful step of outlawing the Java try/finally from
> the Joe-E subset of Java for this reason. There was no other reason to
> outlaw try/finally as there's nothing else inherently unsafe about it. We
> really tried to find another solution but under our constraints -- no
> rewriting of the Java nor change to the JVM -- we could not.
>
> By preventing Joe-E code from catching VirtualMachineErrors and from doing
> a try/finally, the Joe-E code was preemptively terminated immediately on
> occurrence of a VirtualMachineError. Only the spawner of the Joe-E
> computation could react to this termination of the computation it spawned.
>
> This mirrors one of the many thing that Erlang gets right. When a program
> is confused, that program is the last one you want to ask to recover from
> the confusion, since it is already impaired by its own confusion. If you
> don't know what is still true, you are unlikely to engage in repair actions
> correctly. Better to preemptively terminate some large unit containing the
> confusion and recover by
> * restarting from earlier known good state, or
> * if this is not yet feasible, propagating the termination to a yet
> larger granularity of computation.
>
> This is the "fail stop" philosophy of "Death Before Confusion". The
> contrasting philosophy appropriate for some computation is "best efforts".
> Some JavaScript code is best served by one and some by the other. Security
> enforcing code must maintain its own integrity at the price of termination
> (and restart from some coarser grain). Web pages using JavaScript only to
> spice up the user experience are often best served by best efforts. Erlang
> itself is an interesting case study, as its original motivating problem --
> telephone switches -- places a higher priority on uptime than on integrity.
> Nevertheless, both Erlang and the Tandem non-stop architecture found that
> uptime in the large is best served by fail-stop in the small combined with
> coarser-grain recovery logic.
>
> Because JavaScript comes from such a long legacy of de facto best efforts
> architecture, I think a direct du jure shift to fail-stop is unlikely.
> Instead, what we need is a trap-handling mechanism (Erlang "supervisor".
> KeyKOS "keeper"), where different policies can be expressed by user-defined
> trap handlers. When multiple policies co-exist,
>

Sorry for incomplete sentence. Should be

When multiple policies co-exist, the platform obeys the more severe
policies.




> For concreteness, I'll make here a first sketch:
>
> On OOM, the platform first scans the stack to enumerate all realms
> represented by in-progress stack frames as of that moment. (In progress
> meaning that the stack frame still would have been there even if that
> platform had implemented proper-tail-call.) It gathers the trap handlers
> associated with each of those realms. Each trap handler is a pair of a
> string and an optional function.
>
> The string indicates the choice of trap handling strategy, where these
> strategies are ordered

Re: [whatwg] Handling out of memory issues with getImageData/createImageData

2015-09-26 Thread Niels Keurentjes 
> https://esdiscuss.org/topic/using-max-stack-limit-to-determine-current-js-engine-and-revision#content-7
> indicates there may be security issues with throwing out-of-memory exceptions.

That's hardly worth considering. The technique described there for 
fingerprinting is interesting from a theorist's perspective, but exposing no 
data that cannot already be more reliably extracted from navigator.userAgent 
with a simple regex.

An out-of-memory in a sandbox is just exposing information about the sandbox, 
and worth nothing therefore if the sandbox version itself isn’t already 
compromised, at which point the user is generally screwed anyway if he didn't 
patch in time. Being able to detect a vulnerability is not a prerequisite for 
exploiting it.

Niels

-Original Message-
From: whatwg [mailto:whatwg-boun...@lists.whatwg.org] On Behalf Of Anne van 
Kesteren
Sent: zaterdag 26 september 2015 16:35
To: Justin Novosad <ju...@google.com>
Cc: WHAT Working Group <wha...@whatwg.org>; Mark Miller <erig...@gmail.com>
Subject: Re: [whatwg] Handling out of memory issues with 
getImageData/createImageData

On Fri, Sep 25, 2015 at 4:48 PM, Justin Novosad <ju...@google.com> wrote:
> Currently there is no spec'ed behavior for handling out-of memory issues
> for the specific case of attempting to allocate a large buffer through
> image data APIs.

Actually, there is no specified behavior for out-of-memory behavior,
period. This is a problem that starts with the ECMAScript standard and
everything that builds upon it.

I have seen Mark Miller discuss some of the issues surrounding this
and perhaps even the necessity to eventually define it, but so far
this has not happened. Not sure if the full story is documented
somewhere. Mark?

https://esdiscuss.org/topic/using-max-stack-limit-to-determine-current-js-engine-and-revision#content-7
indicates there may be security issues with throwing out-of-memory
exceptions.


-- 
https://annevankesteren.nl/

Re: [whatwg] Handling out of memory issues with getImageData/createImageData

2015-09-26 Thread Rik Cabanier 
On Fri, Sep 25, 2015 at 7:51 AM, Boris Zbarsky  wrote:

> On 9/25/15 10:48 AM, Justin Novosad wrote:
>
>> I am sharing this here in case there would be interest in standardizing
>> this behavior.
>>
>
> I personally think it's a good idea (and throwing an exception is how
> Gecko handles, or at least aims to handle, this situation).


In the past, we discussed that error conditions such as this shouldn't
throw exceptions. Most of the time, this type of error is just temporal and
is resolved in the next frame. Rare exceptions are almost never caught by
the author so the application crashes.

Maybe for out of memory conditions, we could return a fake imagedata object
with nothing but transparent white. In addition an 'isValid' property could
signal if you have a real imagedata object.

Re: [whatwg] Handling out of memory issues with getImageData/createImageData

2015-09-26 Thread Anne van Kesteren 
On Fri, Sep 25, 2015 at 4:48 PM, Justin Novosad  wrote:
> Currently there is no spec'ed behavior for handling out-of memory issues
> for the specific case of attempting to allocate a large buffer through
> image data APIs.

Actually, there is no specified behavior for out-of-memory behavior,
period. This is a problem that starts with the ECMAScript standard and
everything that builds upon it.

I have seen Mark Miller discuss some of the issues surrounding this
and perhaps even the necessity to eventually define it, but so far
this has not happened. Not sure if the full story is documented
somewhere. Mark?

https://esdiscuss.org/topic/using-max-stack-limit-to-determine-current-js-engine-and-revision#content-7
indicates there may be security issues with throwing out-of-memory
exceptions.


-- 
https://annevankesteren.nl/

[whatwg] Handling out of memory issues with getImageData/createImageData

2015-09-25 Thread Justin Novosad 
Hi All,

Currently there is no spec'ed behavior for handling out-of memory issues
for the specific case of attempting to allocate a large buffer through
image data APIs.

In Chrome, the current behavior is to crash the process (out of memory
exception), which results in a "sad browser tab".  We are considering
changing this behavior to throw a DOM exception instead of crashing. The
idea is to provide a better user experience by halting the script rather
than crashing the tab.  In particular, we are often encountering this error
in cases where an advertisement embedded in an iframe is attempting to
allocate a large image data object. By throwing an exception instead of
crashing, only the ad's script context would halt, and the main page would
be able to continue to function normally.

I am sharing this here in case there would be interest in standardizing
this behavior. By standardizing it, developers could expect the exception,
and therefore write more stable apps by calling image data APIs inside a
try/catch scope.

Thoughts?

Justin

Re: [whatwg] Handling out of memory issues with getImageData/createImageData

2015-09-25 Thread Boris Zbarsky 

On 9/25/15 10:48 AM, Justin Novosad wrote:

I am sharing this here in case there would be interest in standardizing
this behavior.


I personally think it's a good idea (and throwing an exception is how 
Gecko handles, or at least aims to handle, this situation).


-Boris