Re: [webkit-dev] XHR responseArrayBuffer attribute: possible implementation

2010-10-25 Thread Geoffrey Garen
Hi Chris.

I like the efficiency of this approach. And I agree with your premise that a 
developer will probably only want one type of data (raw, text, or XML) per 
request, and not more than one.

My biggest concern with this idea is that there's nothing obvious about the API 
pattern of three properties -- .responseText, .responseXML, and 
.responseArrayBuffer -- that makes clear that accessing one should prohibit 
access to the others. I wonder if there's a good way to make this clearer.

Maybe the API should require the programmer to specify in advance what type of 
data he/she will ask for. For example, an extra responeType parameter passed to 
open. The default behavior would be the values currently supported, but you 
could opt into something specific for extra safety/performance, and new types 
of data:

request.open(GET, data.xml, true, Text);
request.open(GET, data.xml, true, XML);
request.open(GET, data.xml, true, Bytes);

Geoff



On Oct 22, 2010, at 4:47 PM, Chris Rogers wrote:

 A few weeks ago I brought up the idea of implementing the responseArrayBuffer 
 attribute for XHR:
 http://dev.w3.org/2006/webapi/XMLHttpRequest-2/#the-responsearraybuffer-attribute
 
 One of the concerns was that it might require double the memory usage since 
 the raw bytes would have to be accumulated along with the decoded text as 
 it's being built up.  One possible solution which I've been discussing with 
 James Robinson and Ken Russell is to defer decoding the text, and instead 
 buffer the raw data as it comes in.  If there's any access to responseText 
 (or responseXML), then the buffered data can be decoded into text at that 
 time, and the buffered raw data discarded.  If that case happens, then from 
 that point on no raw data buffering would happen and the text would be 
 accumulated as it is right now.  Otherwise, if responseText is never accessed 
 then the raw data continues to buffer until it's completely loaded.  Then an 
 access to responseArrayBuffer can easily convert the raw bytes to an 
 ArrayBuffer.
 
 The idea is that once responseText or responseXML is accessed, then it would 
 no longer be possible to access responseArrayBuffer (an exception would be 
 thrown).
 Conversely, once responseArrayBuffer is accessed, then it would no longer be 
 possible to use responseText or responseXML (an exception would be thrown).
 This approach does seem a little strange because of the mutually exclusive 
 nature of the access.  However, it seems that it would be hard to come up for 
 a reasonable use case where both the raw bytes *and* the text would be needed 
 for the same XHR.
 
 How does this sound as an approach?
 
 Chris
 
 
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute: possible implementation

2010-10-25 Thread Darin Fisher
The solution for .responseBlob was to add an .asBlob attribute that would
need to be set to true before calling .send().  We could do the same for
.responseArrayBuffer.

-Darin


On Mon, Oct 25, 2010 at 12:17 PM, Geoffrey Garen gga...@apple.com wrote:

 Hi Chris.

 I like the efficiency of this approach. And I agree with your premise that
 a developer will probably only want one type of data (raw, text, or XML) per
 request, and not more than one.

 My biggest concern with this idea is that there's nothing obvious about the
 API pattern of three properties -- .responseText, .responseXML, and
 .responseArrayBuffer -- that makes clear that accessing one should prohibit
 access to the others. I wonder if there's a good way to make this clearer.

 Maybe the API should require the programmer to specify in advance what type
 of data he/she will ask for. For example, an extra responeType parameter
 passed to open. The default behavior would be the values currently
 supported, but you could opt into something specific for extra
 safety/performance, and new types of data:

 request.open(GET, data.xml, true, Text);
 request.open(GET, data.xml, true, XML);
 request.open(GET, data.xml, true, Bytes);

 Geoff



 On Oct 22, 2010, at 4:47 PM, Chris Rogers wrote:

 A few weeks ago I brought up the idea of implementing the
 responseArrayBuffer attribute for XHR:

 http://dev.w3.org/2006/webapi/XMLHttpRequest-2/#the-responsearraybuffer-attribute

 One of the concerns was that it might require double the memory usage since
 the raw bytes would have to be accumulated along with the decoded text as
 it's being built up.  One possible solution which I've been discussing with
 James Robinson and Ken Russell is to defer decoding the text, and instead
 buffer the raw data as it comes in.  If there's any access to responseText
 (or responseXML), then the buffered data can be decoded into text at that
 time, and the buffered raw data discarded.  If that case happens, then from
 that point on no raw data buffering would happen and the text would be
 accumulated as it is right now.  Otherwise, if responseText is never
 accessed then the raw data continues to buffer until it's completely loaded.
  Then an access to responseArrayBuffer can easily convert the raw bytes to
 an ArrayBuffer.

 The idea is that once responseText or responseXML is accessed, then it
 would no longer be possible to access responseArrayBuffer (an exception
 would be thrown).
 Conversely, once responseArrayBuffer is accessed, then it would no longer
 be possible to use responseText or responseXML (an exception would be
 thrown).
 This approach does seem a little strange because of the mutually exclusive
 nature of the access.  However, it seems that it would be hard to come up
 for a reasonable use case where both the raw bytes *and* the text would be
 needed for the same XHR.

 How does this sound as an approach?

 Chris


 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev



 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute: possible implementation

2010-10-25 Thread Chris Marrin

On Oct 25, 2010, at 12:22 PM, Darin Fisher wrote:

 The solution for .responseBlob was to add an .asBlob attribute that would 
 need to be set to true before calling .send().  We could do the same for 
 .responseArrayBuffer.
 
 -Darin
 
 
 On Mon, Oct 25, 2010 at 12:17 PM, Geoffrey Garen gga...@apple.com wrote:
 Hi Chris.
 
 I like the efficiency of this approach. And I agree with your premise that a 
 developer will probably only want one type of data (raw, text, or XML) per 
 request, and not more than one.
 
 My biggest concern with this idea is that there's nothing obvious about the 
 API pattern of three properties -- .responseText, .responseXML, and 
 .responseArrayBuffer -- that makes clear that accessing one should prohibit 
 access to the others. I wonder if there's a good way to make this clearer.
 
 Maybe the API should require the programmer to specify in advance what type 
 of data he/she will ask for. For example, an extra responeType parameter 
 passed to open. The default behavior would be the values currently supported, 
 but you could opt into something specific for extra safety/performance, and 
 new types of data:
 
 request.open(GET, data.xml, true, Text);
 request.open(GET, data.xml, true, XML);
 request.open(GET, data.xml, true, Bytes);

I'd sure like to try to avoid an explosion in the API. I like Geoff's 
suggestion of specifying the type of request in open(). Seems like the best API 
would be to have Geoff's API and then:

any responseObject();
DOMString responseType();

That would allow us to expand the types supported without any additional API. 
We'd need to support the current API calls for backward compatibility. But now 
seems like a good time to plan for the future.

-
~Chris
cmar...@apple.com




___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute: possible implementation

2010-10-25 Thread Darin Fisher
On Mon, Oct 25, 2010 at 12:34 PM, Chris Marrin cmar...@apple.com wrote:


 On Oct 25, 2010, at 12:22 PM, Darin Fisher wrote:

 The solution for .responseBlob was to add an .asBlob attribute that would
 need to be set to true before calling .send().  We could do the same for
 .responseArrayBuffer.

 -Darin


 On Mon, Oct 25, 2010 at 12:17 PM, Geoffrey Garen gga...@apple.com wrote:

 Hi Chris.

 I like the efficiency of this approach. And I agree with your premise that
 a developer will probably only want one type of data (raw, text, or XML) per
 request, and not more than one.

 My biggest concern with this idea is that there's nothing obvious about
 the API pattern of three properties -- .responseText, .responseXML, and
 .responseArrayBuffer -- that makes clear that accessing one should prohibit
 access to the others. I wonder if there's a good way to make this clearer.

 Maybe the API should require the programmer to specify in advance what
 type of data he/she will ask for. For example, an extra responeType
 parameter passed to open. The default behavior would be the values currently
 supported, but you could opt into something specific for extra
 safety/performance, and new types of data:

 request.open(GET, data.xml, true, Text);
 request.open(GET, data.xml, true, XML);
 request.open(GET, data.xml, true, Bytes);


 I'd sure like to try to avoid an explosion in the API. I like Geoff's
 suggestion of specifying the type of request in open(). Seems like the best
 API would be to have Geoff's API and then:

 any responseObject();
 DOMString responseType();

 That would allow us to expand the types supported without any additional
 API. We'd need to support the current API calls for backward compatibility.
 But now seems like a good time to plan for the future.


Why is that superior to more specifically named attributes and methods?  I'm
trying to see how you would determine at runtime if a browser's XHR
implementation supported returning an ArrayBuffer.

If the XHR had an .asArrayBuffer attribute, then you could test for the
existence of that attribute.

I'm also not sure why adding new response types is cheaper than adding new
attributes / methods.  In both cases, it seems like we have to have a
similar standards discussion.

-Darin



 -
 ~Chris
 cmar...@apple.com





___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute: possible implementation

2010-10-25 Thread Chris Rogers
passing undefined as the 4th and 5th arguments seems pretty clunky to me.
 Since, we already have an asBlob attribute, then asArrayBuffer like
Darin suggests seems like it might be better.  However, then we can get into
cases where both asBlob and asArrayBuffer are set, and this problem
would get even worse as new types are added.  So, an attribute called
responseType might be a good solution.  This would be instead of passing
it as an argument to open(), and instead of having an asBlob attribute.
 This approach seems the cleanest to me, and I'll propose it to the
appropriate standards group.

Chris

On Mon, Oct 25, 2010 at 12:45 PM, Alexey Proskuryakov a...@webkit.org wrote:


 25.10.2010, в 12:34, Chris Marrin написал(а):

 request.open(GET, data.xml, true, Text);
 request.open(GET, data.xml, true, XML);
 request.open(GET, data.xml, true, Bytes);


 I'd sure like to try to avoid an explosion in the API. I like Geoff's
 suggestion of specifying the type of request in open(). Seems like the best
 API would be to have Geoff's API and then:


 Note that open() has username and password as its 4th and 5th arguments.
 So, you'd have to call it like request.open(GET, data.xml, true,
 undefined, undefined, Bytes);

 - WBR, Alexey Proskuryakov


 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute: possible implementation

2010-10-25 Thread Darin Fisher
How do you address the discoverability issue that I raised?  asBlob and
asArrayBuffer have the benefit of being detectable at runtime.  but, a
settable responseType does not support detection of supported values.

-Darin



On Mon, Oct 25, 2010 at 2:54 PM, Chris Rogers crog...@google.com wrote:

 passing undefined as the 4th and 5th arguments seems pretty clunky to me.
  Since, we already have an asBlob attribute, then asArrayBuffer like
 Darin suggests seems like it might be better.  However, then we can get into
 cases where both asBlob and asArrayBuffer are set, and this problem
 would get even worse as new types are added.  So, an attribute called
 responseType might be a good solution.  This would be instead of passing
 it as an argument to open(), and instead of having an asBlob attribute.
  This approach seems the cleanest to me, and I'll propose it to the
 appropriate standards group.

 Chris

 On Mon, Oct 25, 2010 at 12:45 PM, Alexey Proskuryakov a...@webkit.orgwrote:


 25.10.2010, в 12:34, Chris Marrin написал(а):

  request.open(GET, data.xml, true, Text);
 request.open(GET, data.xml, true, XML);
 request.open(GET, data.xml, true, Bytes);


 I'd sure like to try to avoid an explosion in the API. I like Geoff's
 suggestion of specifying the type of request in open(). Seems like the best
 API would be to have Geoff's API and then:


 Note that open() has username and password as its 4th and 5th arguments.
 So, you'd have to call it like request.open(GET, data.xml, true,
 undefined, undefined, Bytes);

  - WBR, Alexey Proskuryakov


 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev



 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute: possible implementation

2010-10-25 Thread Chris Rogers
True, it's not as simple as checking for the existence of an attribute, but
it could throw an exception. It seems like having both asBlob and
asArrayBuffer also creates the possibility of bad combinations of settings
which could get confusing, especially if more types are added in the future.

Chris

On Mon, Oct 25, 2010 at 3:04 PM, Darin Fisher da...@chromium.org wrote:

 How do you address the discoverability issue that I raised?  asBlob and
 asArrayBuffer have the benefit of being detectable at runtime.  but, a
 settable responseType does not support detection of supported values.

 -Darin



 On Mon, Oct 25, 2010 at 2:54 PM, Chris Rogers crog...@google.com wrote:

 passing undefined as the 4th and 5th arguments seems pretty clunky to
 me.  Since, we already have an asBlob attribute, then asArrayBuffer like
 Darin suggests seems like it might be better.  However, then we can get into
 cases where both asBlob and asArrayBuffer are set, and this problem
 would get even worse as new types are added.  So, an attribute called
 responseType might be a good solution.  This would be instead of passing
 it as an argument to open(), and instead of having an asBlob attribute.
  This approach seems the cleanest to me, and I'll propose it to the
 appropriate standards group.

 Chris

 On Mon, Oct 25, 2010 at 12:45 PM, Alexey Proskuryakov a...@webkit.orgwrote:


 25.10.2010, в 12:34, Chris Marrin написал(а):

  request.open(GET, data.xml, true, Text);
 request.open(GET, data.xml, true, XML);
 request.open(GET, data.xml, true, Bytes);


 I'd sure like to try to avoid an explosion in the API. I like Geoff's
 suggestion of specifying the type of request in open(). Seems like the best
 API would be to have Geoff's API and then:


 Note that open() has username and password as its 4th and 5th arguments.
 So, you'd have to call it like request.open(GET, data.xml, true,
 undefined, undefined, Bytes);

  - WBR, Alexey Proskuryakov


 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev



 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev



___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute: possible implementation

2010-10-25 Thread Geoffrey Garen
You could aid discoverability -- and programming -- by providing a set of 
enumerated constants on the XHR constructor, much like the Node constructor 
provides for NodeTypes.

Another option would be to throw an exception when setting responseType to an 
unsupported value.

Geoff

On Oct 25, 2010, at 3:04 PM, Darin Fisher wrote:

 How do you address the discoverability issue that I raised?  asBlob and 
 asArrayBuffer have the benefit of being detectable at runtime.  but, a 
 settable responseType does not support detection of supported values.
 
 -Darin
 
 
 
 On Mon, Oct 25, 2010 at 2:54 PM, Chris Rogers crog...@google.com wrote:
 passing undefined as the 4th and 5th arguments seems pretty clunky to me.  
 Since, we already have an asBlob attribute, then asArrayBuffer like Darin 
 suggests seems like it might be better.  However, then we can get into cases 
 where both asBlob and asArrayBuffer are set, and this problem would get 
 even worse as new types are added.  So, an attribute called responseType 
 might be a good solution.  This would be instead of passing it as an argument 
 to open(), and instead of having an asBlob attribute.  This approach seems 
 the cleanest to me, and I'll propose it to the appropriate standards group.
 
 Chris
 
 On Mon, Oct 25, 2010 at 12:45 PM, Alexey Proskuryakov a...@webkit.org wrote:
 
 25.10.2010, в 12:34, Chris Marrin написал(а):
 
 request.open(GET, data.xml, true, Text);
 request.open(GET, data.xml, true, XML);
 request.open(GET, data.xml, true, Bytes);
 
 I'd sure like to try to avoid an explosion in the API. I like Geoff's 
 suggestion of specifying the type of request in open(). Seems like the best 
 API would be to have Geoff's API and then:
 
 Note that open() has username and password as its 4th and 5th arguments. So, 
 you'd have to call it like request.open(GET, data.xml, true, undefined, 
 undefined, Bytes);
 
 - WBR, Alexey Proskuryakov
 
 
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
 
 
 
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
 
 
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


[webkit-dev] XHR responseArrayBuffer attribute: possible implementation

2010-10-22 Thread Chris Rogers
A few weeks ago I brought up the idea of implementing the
responseArrayBuffer attribute for XHR:
http://dev.w3.org/2006/webapi/XMLHttpRequest-2/#the-responsearraybuffer-attribute

One of the concerns was that it might require double the memory usage since
the raw bytes would have to be accumulated along with the decoded text as
it's being built up.  One possible solution which I've been discussing with
James Robinson and Ken Russell is to defer decoding the text, and instead
buffer the raw data as it comes in.  If there's any access to responseText
(or responseXML), then the buffered data can be decoded into text at that
time, and the buffered raw data discarded.  If that case happens, then from
that point on no raw data buffering would happen and the text would be
accumulated as it is right now.  Otherwise, if responseText is never
accessed then the raw data continues to buffer until it's completely loaded.
 Then an access to responseArrayBuffer can easily convert the raw bytes to
an ArrayBuffer.

The idea is that once responseText or responseXML is accessed, then it would
no longer be possible to access responseArrayBuffer (an exception would be
thrown).
Conversely, once responseArrayBuffer is accessed, then it would no longer be
possible to use responseText or responseXML (an exception would be thrown).
This approach does seem a little strange because of the mutually exclusive
nature of the access.  However, it seems that it would be hard to come up
for a reasonable use case where both the raw bytes *and* the text would be
needed for the same XHR.

How does this sound as an approach?

Chris
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-30 Thread Chris Marrin

On Sep 29, 2010, at 6:34 PM, Maciej Stachowiak wrote:

 
 ...The idea is that when an ArrayBuffer is sent via postMessage, it is
 atomically closed on this side; its publicly visible length goes to
 0, as do the lengths of any views referring to it. On the other side,
 a new ArrayBuffer wrapper object is synthesized, pointing to the same
 storage and with the original length.
 
 To be able to reuse the same memory region over and over again, the
 other side would simply send the ArrayBuffer back for re-filling via
 postMessage. Ping-ponging ArrayBuffers back and forth achieves
 zero-copy transfer of large amounts of data while still maintaining
 the shared-nothing semantic. The only allocations are for the (tiny)
 ArrayBuffer wrapper objects, but the underlying storage is stable.
 
 Implementing this idea will require a couple of minor additions to the
 TypedArray specification (in particular, the addition of a close
 method on ArrayBuffer) as well as defining the semantics of sending an
 ArrayBuffer via postMessage. I hope to prototype it soon.
 
 Regarding your scenario, I would simply post the ArrayBuffer from the
 XHR object to the worker with the above semantics. The main thread
 would then not be able to access the data in the ArrayBuffer, but
 sending it to the worker for processing would not involve any data
 copies.
 
 Sure, transfer semantics avoid shared mutable state, though it would be 
 inconsistent with most other pure data types. But what if you have some 
 data that doesn't need mutating but you'd like to share with multiple other 
 Workers? Now you'd be forced to explicitly copy. The availability of an 
 immutable variant would let you avoid that. At most, you'd need to copy once 
 if your ArrayBuffer started immutable; or you could have the ability to 
 convert mutable to immutable at runtime (it would have to be a one-way 
 conversion, of course).


I'm thinking about how this would be implemented. Ken talks about a close 
function to make it possible to pass an ArrayBuffer to a worker. If I have it 
right, this would detach the contents of the ArrayBuffer from it's owning 
object, replacing it with a 0 length buffer. Then the worker attaches the 
contents to a new ArrayBuffer owned by the worker. To do that we'd need to 
figure out the magic of passing this bare buffer to the worker. An 
ImmutableArrayBuffer would not need any such magic. But without any additional 
functionality, you'd always need an additional copy (even it's a copy-on-write) 
for Maciej's example.

In Maciej's example, he wants to take an incoming buffer and pass it to a 
worker, presumably so it can be mutated into something else. So you'd pass the 
ImmutableArrayBuffer to the worker (no copy) and it would create a new 
ArrayBuffer with one or more views which it would fill with the mutated data. 
But to pass this buffer back to the main thread, you'd need to convert this 
ArrayBuffer to an ImmutableArrayBuffer, which would require some sort of copy. 

What's needed is a way to pass that ArrayBuffer back to the main thread without 
a copy. So maybe we just need a function like Ken's close but without the 
magic. A makeImmutable() function could be called on the ArrayBuffer, which 
would create a new ImmutableArrayBuffer, attach the contents of the ArrayBuffer 
to it and set the contents of the ArrayBuffer to a 0 length buffer, as in Ken's 
design.

So now you'd pass the incoming ImmutableArrayBuffer to the worker, create a new 
ArrayBuffer for the mutated data, fill it, call makeImmutable on it and return 
the result. No copies would be needed. Once the process starts, the old buffers 
can be recycled to avoid memory allocations as well.

Would something like that work?

-
~Chris
cmar...@apple.com




___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-30 Thread Kenneth Russell
On Thu, Sep 30, 2010 at 7:09 AM, Chris Marrin cmar...@apple.com wrote:

 On Sep 29, 2010, at 6:34 PM, Maciej Stachowiak wrote:

 ...The idea is that when an ArrayBuffer is sent via postMessage, it is

 atomically closed on this side; its publicly visible length goes to

 0, as do the lengths of any views referring to it. On the other side,

 a new ArrayBuffer wrapper object is synthesized, pointing to the same

 storage and with the original length.

 To be able to reuse the same memory region over and over again, the

 other side would simply send the ArrayBuffer back for re-filling via

 postMessage. Ping-ponging ArrayBuffers back and forth achieves

 zero-copy transfer of large amounts of data while still maintaining

 the shared-nothing semantic. The only allocations are for the (tiny)

 ArrayBuffer wrapper objects, but the underlying storage is stable.

 Implementing this idea will require a couple of minor additions to the

 TypedArray specification (in particular, the addition of a close

 method on ArrayBuffer) as well as defining the semantics of sending an

 ArrayBuffer via postMessage. I hope to prototype it soon.

 Regarding your scenario, I would simply post the ArrayBuffer from the

 XHR object to the worker with the above semantics. The main thread

 would then not be able to access the data in the ArrayBuffer, but

 sending it to the worker for processing would not involve any data

 copies.

 Sure, transfer semantics avoid shared mutable state, though it would be
 inconsistent with most other pure data types. But what if you have some
 data that doesn't need mutating but you'd like to share with multiple other
 Workers? Now you'd be forced to explicitly copy. The availability of an
 immutable variant would let you avoid that. At most, you'd need to copy once
 if your ArrayBuffer started immutable; or you could have the ability to
 convert mutable to immutable at runtime (it would have to be a one-way
 conversion, of course).

 I'm thinking about how this would be implemented. Ken talks about a close
 function to make it possible to pass an ArrayBuffer to a worker. If I have
 it right, this would detach the contents of the ArrayBuffer from it's owning
 object, replacing it with a 0 length buffer. Then the worker attaches the
 contents to a new ArrayBuffer owned by the worker. To do that we'd need to
 figure out the magic of passing this bare buffer to the worker. An
 ImmutableArrayBuffer would not need any such magic. But without any
 additional functionality, you'd always need an additional copy (even it's a
 copy-on-write) for Maciej's example.
 In Maciej's example, he wants to take an incoming buffer and pass it to a
 worker, presumably so it can be mutated into something else. So you'd pass
 the ImmutableArrayBuffer to the worker (no copy) and it would create a new
 ArrayBuffer with one or more views which it would fill with the mutated
 data. But to pass this buffer back to the main thread, you'd need to convert
 this ArrayBuffer to an ImmutableArrayBuffer, which would require some sort
 of copy.
 What's needed is a way to pass that ArrayBuffer back to the main thread
 without a copy. So maybe we just need a function like Ken's close but
 without the magic. A makeImmutable() function could be called on the
 ArrayBuffer, which would create a new ImmutableArrayBuffer, attach the
 contents of the ArrayBuffer to it and set the contents of the ArrayBuffer to
 a 0 length buffer, as in Ken's design.
 So now you'd pass the incoming ImmutableArrayBuffer to the worker, create a
 new ArrayBuffer for the mutated data, fill it, call makeImmutable on it and
 return the result. No copies would be needed. Once the process starts, the
 old buffers can be recycled to avoid memory allocations as well.
 Would something like that work?

I can see the need both for immutable data and transfer semantics. I
don't think that adding a new type (ImmutableArrayBuffer) is the right
way to do it, because it significantly complicates the type hierarchy.
Rather, I think immutability should be a read-only property on the
ArrayBuffer, set at creation time, and affecting the kinds of views
that can be attached to it. I'll raise the issue and a proposal on the
public_webgl mailing list.

-Ken
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-30 Thread Chris Marrin

On Sep 30, 2010, at 10:46 AM, Kenneth Russell wrote:

 ...
 
 Sure, transfer semantics avoid shared mutable state, though it would be
 inconsistent with most other pure data types. But what if you have some
 data that doesn't need mutating but you'd like to share with multiple other
 Workers? Now you'd be forced to explicitly copy. The availability of an
 immutable variant would let you avoid that. At most, you'd need to copy once
 if your ArrayBuffer started immutable; or you could have the ability to
 convert mutable to immutable at runtime (it would have to be a one-way
 conversion, of course).
 
 I'm thinking about how this would be implemented. Ken talks about a close
 function to make it possible to pass an ArrayBuffer to a worker. If I have
 it right, this would detach the contents of the ArrayBuffer from it's owning
 object, replacing it with a 0 length buffer. Then the worker attaches the
 contents to a new ArrayBuffer owned by the worker. To do that we'd need to
 figure out the magic of passing this bare buffer to the worker. An
 ImmutableArrayBuffer would not need any such magic. But without any
 additional functionality, you'd always need an additional copy (even it's a
 copy-on-write) for Maciej's example.
 In Maciej's example, he wants to take an incoming buffer and pass it to a
 worker, presumably so it can be mutated into something else. So you'd pass
 the ImmutableArrayBuffer to the worker (no copy) and it would create a new
 ArrayBuffer with one or more views which it would fill with the mutated
 data. But to pass this buffer back to the main thread, you'd need to convert
 this ArrayBuffer to an ImmutableArrayBuffer, which would require some sort
 of copy.
 What's needed is a way to pass that ArrayBuffer back to the main thread
 without a copy. So maybe we just need a function like Ken's close but
 without the magic. A makeImmutable() function could be called on the
 ArrayBuffer, which would create a new ImmutableArrayBuffer, attach the
 contents of the ArrayBuffer to it and set the contents of the ArrayBuffer to
 a 0 length buffer, as in Ken's design.
 So now you'd pass the incoming ImmutableArrayBuffer to the worker, create a
 new ArrayBuffer for the mutated data, fill it, call makeImmutable on it and
 return the result. No copies would be needed. Once the process starts, the
 old buffers can be recycled to avoid memory allocations as well.
 Would something like that work?
 
 I can see the need both for immutable data and transfer semantics. I
 don't think that adding a new type (ImmutableArrayBuffer) is the right
 way to do it, because it significantly complicates the type hierarchy.
 Rather, I think immutability should be a read-only property on the
 ArrayBuffer, set at creation time, and affecting the kinds of views
 that can be attached to it. I'll raise the issue and a proposal on the
 public_webgl mailing list.

There are many ways to do it. If we do it as a read-only property, then we need 
to do a write check on every access. Doing it as a completely set of immutable 
classes (ArrayBuffer and views) would double the number of classes. But there 
are only 9 classes now, so the increase wouldn't be that bad. This is 
especially true with the way the spec is now. All the views are collapsed into 
a single section. So we're really just talking about adding 2 new sections, 
plus a description of the semantics, the new makeImmutable() function on 
ArrayBuffer and probably some copy functions.

-
~Chris
cmar...@apple.com




___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-30 Thread Kenneth Russell
On Thu, Sep 30, 2010 at 11:51 AM, Chris Marrin cmar...@apple.com wrote:

 On Sep 30, 2010, at 10:46 AM, Kenneth Russell wrote:

 ...

 Sure, transfer semantics avoid shared mutable state, though it would be
 inconsistent with most other pure data types. But what if you have some
 data that doesn't need mutating but you'd like to share with multiple other
 Workers? Now you'd be forced to explicitly copy. The availability of an
 immutable variant would let you avoid that. At most, you'd need to copy once
 if your ArrayBuffer started immutable; or you could have the ability to
 convert mutable to immutable at runtime (it would have to be a one-way
 conversion, of course).

 I'm thinking about how this would be implemented. Ken talks about a close
 function to make it possible to pass an ArrayBuffer to a worker. If I have
 it right, this would detach the contents of the ArrayBuffer from it's owning
 object, replacing it with a 0 length buffer. Then the worker attaches the
 contents to a new ArrayBuffer owned by the worker. To do that we'd need to
 figure out the magic of passing this bare buffer to the worker. An
 ImmutableArrayBuffer would not need any such magic. But without any
 additional functionality, you'd always need an additional copy (even it's a
 copy-on-write) for Maciej's example.
 In Maciej's example, he wants to take an incoming buffer and pass it to a
 worker, presumably so it can be mutated into something else. So you'd pass
 the ImmutableArrayBuffer to the worker (no copy) and it would create a new
 ArrayBuffer with one or more views which it would fill with the mutated
 data. But to pass this buffer back to the main thread, you'd need to convert
 this ArrayBuffer to an ImmutableArrayBuffer, which would require some sort
 of copy.
 What's needed is a way to pass that ArrayBuffer back to the main thread
 without a copy. So maybe we just need a function like Ken's close but
 without the magic. A makeImmutable() function could be called on the
 ArrayBuffer, which would create a new ImmutableArrayBuffer, attach the
 contents of the ArrayBuffer to it and set the contents of the ArrayBuffer to
 a 0 length buffer, as in Ken's design.
 So now you'd pass the incoming ImmutableArrayBuffer to the worker, create a
 new ArrayBuffer for the mutated data, fill it, call makeImmutable on it and
 return the result. No copies would be needed. Once the process starts, the
 old buffers can be recycled to avoid memory allocations as well.
 Would something like that work?

 I can see the need both for immutable data and transfer semantics. I
 don't think that adding a new type (ImmutableArrayBuffer) is the right
 way to do it, because it significantly complicates the type hierarchy.
 Rather, I think immutability should be a read-only property on the
 ArrayBuffer, set at creation time, and affecting the kinds of views
 that can be attached to it. I'll raise the issue and a proposal on the
 public_webgl mailing list.

 There are many ways to do it. If we do it as a read-only property, then we 
 need to do a write check on every access.

This is incorrect. The view hierarchy can be split into read-only and
read-write versions, and attachment of a read-write view to a
read-only ArrayBuffer specified to throw an exception. Splitting
ArrayBuffer into two classes is much more traumatic.

We should probably continue this discussion on public_webgl since I
suspect it isn't of interest to the majority of webkit-dev
subscribers.

-Ken

 Doing it as a completely set of immutable classes (ArrayBuffer and views) 
 would double the number of classes. But there are only 9 classes now, so the 
 increase wouldn't be that bad. This is especially true with the way the spec 
 is now. All the views are collapsed into a single section. So we're really 
 just talking about adding 2 new sections, plus a description of the 
 semantics, the new makeImmutable() function on ArrayBuffer and probably some 
 copy functions.

 -
 ~Chris
 cmar...@apple.com





___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-29 Thread Kenneth Russell
On Tue, Sep 28, 2010 at 11:26 AM, Maciej Stachowiak m...@apple.com wrote:

 On Sep 28, 2010, at 11:05 AM, Kenneth Russell wrote:

 On Tue, Sep 28, 2010 at 9:45 AM, Maciej Stachowiak m...@apple.com wrote:

 On Sep 28, 2010, at 7:15 AM, Chris Marrin wrote:


 On Sep 27, 2010, at 6:37 PM, Maciej Stachowiak wrote:


 On Sep 27, 2010, at 3:19 PM, Michael Nordman wrote:

 Webkit's XHR currently does not keep two copies of the data that I can 
 see. I think we should avoid that.

 We could keep the raw data around, which hopefully is directly usable as 
 an ArrayBuffer backing store, and only translate it to text format 
 when/if the client requests responseText.

 Yes, the raw data should be usable without translation in an ArrayBuffer. 
 But we'd still need to make a copy of the raw bits when a new ArrayBuffer 
 is created via responseArrayBuffer(), because that object is mutable.

 Is there an immutable variant of ArrayBuffer? If not, we really need one. 
 But even without that, note that you don't necessarily need to make an 
 immediate copy, you can use copy-on-write.

 The immutable variant would be helpful since we could avoid implementing 
 threadsafe copy-on-write just to allow efficient passing of ArrayBuffers to 
 Workers.

 Chris has raised this issue on the public_webgl list, and we've begun
 discussion there, but I would like to point out that having an
 immutable ArrayBuffer and views on it does not help with the situation
 of passing data to or from a web worker. The side that constructs the
 data will necessarily have a mutable view, so it will be able to cause
 changes that can be seen on the other side even if the view on the
 other side is immutable.

 Not if the side that got the data got it in immutable form in the first 
 place. For example, if you get an immutable ArrayBuffer from XHR in a Worker, 
 then you can pass it to another Worker or to the main thread without the need 
 for any copying or copy-on-write.


 We have a design that will allow efficient zero-copy producer/consumer
 queues to be implemented using TypedArrays while maintaining
 ECMAScript's shared-nothing semantics. I'll be happy to sketch it out,
 but it's probably most appropriate for a mailing list like
 public_webgl.

 I'm curious to hear it and I don't follow public_webgl.

 I'd specifically like to handle the following case:
 - You obtain a chunk of binary data from the network or filesystem and want 
 to pass it to another thread without copying.

The scenario to which I've given the most thought is that where a
continuous stream of data is sent from a worker to the main thread,
vice versa, or back and forth. I'll describe the solution for this
case first and then discuss how it applies to yours.

In the producer/consumer scenario it is essential to avoid continuous
memory allocation and deallocation. Ideally the main thread and worker
would share a fixed size memory region. Since this would violate the
shared-nothing semantic, a different solution is needed.

The idea is that when an ArrayBuffer is sent via postMessage, it is
atomically closed on this side; its publicly visible length goes to
0, as do the lengths of any views referring to it. On the other side,
a new ArrayBuffer wrapper object is synthesized, pointing to the same
storage and with the original length.

To be able to reuse the same memory region over and over again, the
other side would simply send the ArrayBuffer back for re-filling via
postMessage. Ping-ponging ArrayBuffers back and forth achieves
zero-copy transfer of large amounts of data while still maintaining
the shared-nothing semantic. The only allocations are for the (tiny)
ArrayBuffer wrapper objects, but the underlying storage is stable.

Implementing this idea will require a couple of minor additions to the
TypedArray specification (in particular, the addition of a close
method on ArrayBuffer) as well as defining the semantics of sending an
ArrayBuffer via postMessage. I hope to prototype it soon.

Regarding your scenario, I would simply post the ArrayBuffer from the
XHR object to the worker with the above semantics. The main thread
would then not be able to access the data in the ArrayBuffer, but
sending it to the worker for processing would not involve any data
copies.

I don't understand why the ArrayBuffer returned from the XHR needs to
be strictly immutable. Since the application created the XHR object,
it seems to me it should be fine if it mutates the copy of the data
attached to it.

-Ken
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-29 Thread Maciej Stachowiak

On Sep 29, 2010, at 2:02 PM, Kenneth Russell wrote:

 On Tue, Sep 28, 2010 at 11:26 AM, Maciej Stachowiak m...@apple.com wrote:
 
 On Sep 28, 2010, at 11:05 AM, Kenneth Russell wrote:
 
 On Tue, Sep 28, 2010 at 9:45 AM, Maciej Stachowiak m...@apple.com wrote:
 
 On Sep 28, 2010, at 7:15 AM, Chris Marrin wrote:
 
 
 On Sep 27, 2010, at 6:37 PM, Maciej Stachowiak wrote:
 
 
 On Sep 27, 2010, at 3:19 PM, Michael Nordman wrote:
 
 Webkit's XHR currently does not keep two copies of the data that I can 
 see. I think we should avoid that.
 
 We could keep the raw data around, which hopefully is directly usable as 
 an ArrayBuffer backing store, and only translate it to text format 
 when/if the client requests responseText.
 
 Yes, the raw data should be usable without translation in an ArrayBuffer. 
 But we'd still need to make a copy of the raw bits when a new ArrayBuffer 
 is created via responseArrayBuffer(), because that object is mutable.
 
 Is there an immutable variant of ArrayBuffer? If not, we really need one. 
 But even without that, note that you don't necessarily need to make an 
 immediate copy, you can use copy-on-write.
 
 The immutable variant would be helpful since we could avoid implementing 
 threadsafe copy-on-write just to allow efficient passing of ArrayBuffers 
 to Workers.
 
 Chris has raised this issue on the public_webgl list, and we've begun
 discussion there, but I would like to point out that having an
 immutable ArrayBuffer and views on it does not help with the situation
 of passing data to or from a web worker. The side that constructs the
 data will necessarily have a mutable view, so it will be able to cause
 changes that can be seen on the other side even if the view on the
 other side is immutable.
 
 Not if the side that got the data got it in immutable form in the first 
 place. For example, if you get an immutable ArrayBuffer from XHR in a 
 Worker, then you can pass it to another Worker or to the main thread without 
 the need for any copying or copy-on-write.
 
 
 We have a design that will allow efficient zero-copy producer/consumer
 queues to be implemented using TypedArrays while maintaining
 ECMAScript's shared-nothing semantics. I'll be happy to sketch it out,
 but it's probably most appropriate for a mailing list like
 public_webgl.
 
 I'm curious to hear it and I don't follow public_webgl.
 
 I'd specifically like to handle the following case:
 - You obtain a chunk of binary data from the network or filesystem and want 
 to pass it to another thread without copying.
 
 The scenario to which I've given the most thought is that where a
 continuous stream of data is sent from a worker to the main thread,
 vice versa, or back and forth. I'll describe the solution for this
 case first and then discuss how it applies to yours.
 
 In the producer/consumer scenario it is essential to avoid continuous
 memory allocation and deallocation. Ideally the main thread and worker
 would share a fixed size memory region. Since this would violate the
 shared-nothing semantic, a different solution is needed.
 
 The idea is that when an ArrayBuffer is sent via postMessage, it is
 atomically closed on this side; its publicly visible length goes to
 0, as do the lengths of any views referring to it. On the other side,
 a new ArrayBuffer wrapper object is synthesized, pointing to the same
 storage and with the original length.
 
 To be able to reuse the same memory region over and over again, the
 other side would simply send the ArrayBuffer back for re-filling via
 postMessage. Ping-ponging ArrayBuffers back and forth achieves
 zero-copy transfer of large amounts of data while still maintaining
 the shared-nothing semantic. The only allocations are for the (tiny)
 ArrayBuffer wrapper objects, but the underlying storage is stable.
 
 Implementing this idea will require a couple of minor additions to the
 TypedArray specification (in particular, the addition of a close
 method on ArrayBuffer) as well as defining the semantics of sending an
 ArrayBuffer via postMessage. I hope to prototype it soon.
 
 Regarding your scenario, I would simply post the ArrayBuffer from the
 XHR object to the worker with the above semantics. The main thread
 would then not be able to access the data in the ArrayBuffer, but
 sending it to the worker for processing would not involve any data
 copies.

Sure, transfer semantics avoid shared mutable state, though it would be 
inconsistent with most other pure data types. But what if you have some data 
that doesn't need mutating but you'd like to share with multiple other Workers? 
Now you'd be forced to explicitly copy. The availability of an immutable 
variant would let you avoid that. At most, you'd need to copy once if your 
ArrayBuffer started immutable; or you could have the ability to convert mutable 
to immutable at runtime (it would have to be a one-way conversion, of course).

 
 I don't understand why the ArrayBuffer returned from the XHR needs to
 be 

Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-29 Thread Eric Uhrhane
On Wed, Sep 29, 2010 at 6:34 PM, Maciej Stachowiak m...@apple.com wrote:

 On Sep 29, 2010, at 2:02 PM, Kenneth Russell wrote:

 On Tue, Sep 28, 2010 at 11:26 AM, Maciej Stachowiak m...@apple.com wrote:

 On Sep 28, 2010, at 11:05 AM, Kenneth Russell wrote:

 On Tue, Sep 28, 2010 at 9:45 AM, Maciej Stachowiak m...@apple.com wrote:

 On Sep 28, 2010, at 7:15 AM, Chris Marrin wrote:


 On Sep 27, 2010, at 6:37 PM, Maciej Stachowiak wrote:


 On Sep 27, 2010, at 3:19 PM, Michael Nordman wrote:

 Webkit's XHR currently does not keep two copies of the data that I can 
 see. I think we should avoid that.

 We could keep the raw data around, which hopefully is directly usable 
 as an ArrayBuffer backing store, and only translate it to text format 
 when/if the client requests responseText.

 Yes, the raw data should be usable without translation in an 
 ArrayBuffer. But we'd still need to make a copy of the raw bits when a 
 new ArrayBuffer is created via responseArrayBuffer(), because that 
 object is mutable.

 Is there an immutable variant of ArrayBuffer? If not, we really need one. 
 But even without that, note that you don't necessarily need to make an 
 immediate copy, you can use copy-on-write.

 The immutable variant would be helpful since we could avoid implementing 
 threadsafe copy-on-write just to allow efficient passing of ArrayBuffers 
 to Workers.

 Chris has raised this issue on the public_webgl list, and we've begun
 discussion there, but I would like to point out that having an
 immutable ArrayBuffer and views on it does not help with the situation
 of passing data to or from a web worker. The side that constructs the
 data will necessarily have a mutable view, so it will be able to cause
 changes that can be seen on the other side even if the view on the
 other side is immutable.

 Not if the side that got the data got it in immutable form in the first 
 place. For example, if you get an immutable ArrayBuffer from XHR in a 
 Worker, then you can pass it to another Worker or to the main thread 
 without the need for any copying or copy-on-write.


 We have a design that will allow efficient zero-copy producer/consumer
 queues to be implemented using TypedArrays while maintaining
 ECMAScript's shared-nothing semantics. I'll be happy to sketch it out,
 but it's probably most appropriate for a mailing list like
 public_webgl.

 I'm curious to hear it and I don't follow public_webgl.

 I'd specifically like to handle the following case:
 - You obtain a chunk of binary data from the network or filesystem and want 
 to pass it to another thread without copying.

 The scenario to which I've given the most thought is that where a
 continuous stream of data is sent from a worker to the main thread,
 vice versa, or back and forth. I'll describe the solution for this
 case first and then discuss how it applies to yours.

 In the producer/consumer scenario it is essential to avoid continuous
 memory allocation and deallocation. Ideally the main thread and worker
 would share a fixed size memory region. Since this would violate the
 shared-nothing semantic, a different solution is needed.

 The idea is that when an ArrayBuffer is sent via postMessage, it is
 atomically closed on this side; its publicly visible length goes to
 0, as do the lengths of any views referring to it. On the other side,
 a new ArrayBuffer wrapper object is synthesized, pointing to the same
 storage and with the original length.

 To be able to reuse the same memory region over and over again, the
 other side would simply send the ArrayBuffer back for re-filling via
 postMessage. Ping-ponging ArrayBuffers back and forth achieves
 zero-copy transfer of large amounts of data while still maintaining
 the shared-nothing semantic. The only allocations are for the (tiny)
 ArrayBuffer wrapper objects, but the underlying storage is stable.

 Implementing this idea will require a couple of minor additions to the
 TypedArray specification (in particular, the addition of a close
 method on ArrayBuffer) as well as defining the semantics of sending an
 ArrayBuffer via postMessage. I hope to prototype it soon.

 Regarding your scenario, I would simply post the ArrayBuffer from the
 XHR object to the worker with the above semantics. The main thread
 would then not be able to access the data in the ArrayBuffer, but
 sending it to the worker for processing would not involve any data
 copies.

 Sure, transfer semantics avoid shared mutable state, though it would be 
 inconsistent with most other pure data types. But what if you have some 
 data that doesn't need mutating but you'd like to share with multiple other 
 Workers? Now you'd be forced to explicitly copy. The availability of an 
 immutable variant would let you avoid that. At most, you'd need to copy once 
 if your ArrayBuffer started immutable; or you could have the ability to 
 convert mutable to immutable at runtime (it would have to be a one-way 
 conversion, of course).


 I don't 

Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-28 Thread Chris Marrin

On Sep 27, 2010, at 6:37 PM, Maciej Stachowiak wrote:

 
 On Sep 27, 2010, at 3:19 PM, Michael Nordman wrote:
 
 Webkit's XHR currently does not keep two copies of the data that I can see. 
 I think we should avoid that.
 
 We could keep the raw data around, which hopefully is directly usable as an 
 ArrayBuffer backing store, and only translate it to text format when/if the 
 client requests responseText. 

Yes, the raw data should be usable without translation in an ArrayBuffer. But 
we'd still need to make a copy of the raw bits when a new ArrayBuffer is 
created via responseArrayBuffer(), because that object is mutable.

-
~Chris
cmar...@apple.com




___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-28 Thread Chris Marrin

On Sep 27, 2010, at 6:40 PM, James Robinson wrote:

 On Mon, Sep 27, 2010 at 6:37 PM, Maciej Stachowiak m...@apple.com wrote:
 
 On Sep 27, 2010, at 3:19 PM, Michael Nordman wrote:
 
 Webkit's XHR currently does not keep two copies of the data that I can see. 
 I think we should avoid that.
 
 We could keep the raw data around, which hopefully is directly usable as an 
 ArrayBuffer backing store, and only translate it to text format when/if the 
 client requests responseText. 
 
 
 It would be unfortunate to have to keep the raw data around after the page 
 access .responseText, given that the overwhelming majority of pages will 
 touch .responseText and nothing else.  I found when improving the V8 XHR 
 implementation that the memory footprint of .responseText being held off of 
 the XHR object itself was often significant so I would be very reluctant to 
 grow it by an addition 50-100% (depending on encoding) in the common case.

But do you think you'd ever need more than one copy of the raw bits from the 
response? Seems like you should be able to return a responseText and a 
responseArrayBuffer from the same raw bits. Am I missing some detail of how XHR 
works?

-
~Chris
cmar...@apple.com




___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-28 Thread Maciej Stachowiak

On Sep 28, 2010, at 7:15 AM, Chris Marrin wrote:

 
 On Sep 27, 2010, at 6:37 PM, Maciej Stachowiak wrote:
 
 
 On Sep 27, 2010, at 3:19 PM, Michael Nordman wrote:
 
 Webkit's XHR currently does not keep two copies of the data that I can see. 
 I think we should avoid that.
 
 We could keep the raw data around, which hopefully is directly usable as an 
 ArrayBuffer backing store, and only translate it to text format when/if the 
 client requests responseText. 
 
 Yes, the raw data should be usable without translation in an ArrayBuffer. But 
 we'd still need to make a copy of the raw bits when a new ArrayBuffer is 
 created via responseArrayBuffer(), because that object is mutable.

Is there an immutable variant of ArrayBuffer? If not, we really need one. But 
even without that, note that you don't necessarily need to make an immediate 
copy, you can use copy-on-write.

The immutable variant would be helpful since we could avoid implementing 
threadsafe copy-on-write just to allow efficient passing of ArrayBuffers to 
Workers.

Regards,
Maciej

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-28 Thread Chris Marrin

On Sep 28, 2010, at 9:45 AM, Maciej Stachowiak wrote:

 
 On Sep 28, 2010, at 7:15 AM, Chris Marrin wrote:
 
 
 On Sep 27, 2010, at 6:37 PM, Maciej Stachowiak wrote:
 
 
 On Sep 27, 2010, at 3:19 PM, Michael Nordman wrote:
 
 Webkit's XHR currently does not keep two copies of the data that I can 
 see. I think we should avoid that.
 
 We could keep the raw data around, which hopefully is directly usable as an 
 ArrayBuffer backing store, and only translate it to text format when/if the 
 client requests responseText. 
 
 Yes, the raw data should be usable without translation in an ArrayBuffer. 
 But we'd still need to make a copy of the raw bits when a new ArrayBuffer is 
 created via responseArrayBuffer(), because that object is mutable.
 
 Is there an immutable variant of ArrayBuffer? If not, we really need one. But 
 even without that, note that you don't necessarily need to make an immediate 
 copy, you can use copy-on-write.
 
 The immutable variant would be helpful since we could avoid implementing 
 threadsafe copy-on-write just to allow efficient passing of ArrayBuffers to 
 Workers.

There's not an immutable variant, but that's a good idea. I will propose it

-
~Chris
cmar...@apple.com




___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-28 Thread Kenneth Russell
On Tue, Sep 28, 2010 at 9:45 AM, Maciej Stachowiak m...@apple.com wrote:

 On Sep 28, 2010, at 7:15 AM, Chris Marrin wrote:


 On Sep 27, 2010, at 6:37 PM, Maciej Stachowiak wrote:


 On Sep 27, 2010, at 3:19 PM, Michael Nordman wrote:

 Webkit's XHR currently does not keep two copies of the data that I can 
 see. I think we should avoid that.

 We could keep the raw data around, which hopefully is directly usable as an 
 ArrayBuffer backing store, and only translate it to text format when/if the 
 client requests responseText.

 Yes, the raw data should be usable without translation in an ArrayBuffer. 
 But we'd still need to make a copy of the raw bits when a new ArrayBuffer is 
 created via responseArrayBuffer(), because that object is mutable.

 Is there an immutable variant of ArrayBuffer? If not, we really need one. But 
 even without that, note that you don't necessarily need to make an immediate 
 copy, you can use copy-on-write.

 The immutable variant would be helpful since we could avoid implementing 
 threadsafe copy-on-write just to allow efficient passing of ArrayBuffers to 
 Workers.

Chris has raised this issue on the public_webgl list, and we've begun
discussion there, but I would like to point out that having an
immutable ArrayBuffer and views on it does not help with the situation
of passing data to or from a web worker. The side that constructs the
data will necessarily have a mutable view, so it will be able to cause
changes that can be seen on the other side even if the view on the
other side is immutable.

We have a design that will allow efficient zero-copy producer/consumer
queues to be implemented using TypedArrays while maintaining
ECMAScript's shared-nothing semantics. I'll be happy to sketch it out,
but it's probably most appropriate for a mailing list like
public_webgl.

-Ken
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-28 Thread Maciej Stachowiak

On Sep 28, 2010, at 11:05 AM, Kenneth Russell wrote:

 On Tue, Sep 28, 2010 at 9:45 AM, Maciej Stachowiak m...@apple.com wrote:
 
 On Sep 28, 2010, at 7:15 AM, Chris Marrin wrote:
 
 
 On Sep 27, 2010, at 6:37 PM, Maciej Stachowiak wrote:
 
 
 On Sep 27, 2010, at 3:19 PM, Michael Nordman wrote:
 
 Webkit's XHR currently does not keep two copies of the data that I can 
 see. I think we should avoid that.
 
 We could keep the raw data around, which hopefully is directly usable as 
 an ArrayBuffer backing store, and only translate it to text format when/if 
 the client requests responseText.
 
 Yes, the raw data should be usable without translation in an ArrayBuffer. 
 But we'd still need to make a copy of the raw bits when a new ArrayBuffer 
 is created via responseArrayBuffer(), because that object is mutable.
 
 Is there an immutable variant of ArrayBuffer? If not, we really need one. 
 But even without that, note that you don't necessarily need to make an 
 immediate copy, you can use copy-on-write.
 
 The immutable variant would be helpful since we could avoid implementing 
 threadsafe copy-on-write just to allow efficient passing of ArrayBuffers to 
 Workers.
 
 Chris has raised this issue on the public_webgl list, and we've begun
 discussion there, but I would like to point out that having an
 immutable ArrayBuffer and views on it does not help with the situation
 of passing data to or from a web worker. The side that constructs the
 data will necessarily have a mutable view, so it will be able to cause
 changes that can be seen on the other side even if the view on the
 other side is immutable.

Not if the side that got the data got it in immutable form in the first place. 
For example, if you get an immutable ArrayBuffer from XHR in a Worker, then you 
can pass it to another Worker or to the main thread without the need for any 
copying or copy-on-write.

 
 We have a design that will allow efficient zero-copy producer/consumer
 queues to be implemented using TypedArrays while maintaining
 ECMAScript's shared-nothing semantics. I'll be happy to sketch it out,
 but it's probably most appropriate for a mailing list like
 public_webgl.

I'm curious to hear it and I don't follow public_webgl.

I'd specifically like to handle the following case:
- You obtain a chunk of binary data from the network or filesystem and want to 
pass it to another thread without copying.

Regards,
Maciej

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-27 Thread Michael Nordman
Yes, if we go with telling xhr up front for the array buffer case, I guess
an enum would be slightly better.

On Fri, Sep 24, 2010 at 8:39 PM, Chris Rogers crog...@google.com wrote:

 If we added xhr.asArrayBuffer, what would happen if xhr.asBlob was also
 set?  Don't we really want something like xhr.loadAsType with different enum
 values for text, blob, array buffer, etc.?

 On Fri, Sep 24, 2010 at 5:19 PM, Michael Nordman micha...@google.comwrote:

 With xhr.responseBlob we chose to have the caller decide up front and tell
 the xhr object how it would like the response by setting the xhr.asBlob
 attribute prior to calling send(). We could do the same with
 xhr.asArrayBuffer.

 On Fri, Sep 24, 2010 at 5:09 PM, Alexey Proskuryakov a...@webkit.orgwrote:


 24.09.2010, в 16:37, Chris Rogers написал(а):

  I was interested to know if anybody was planning on implementing that
 attribute soon.  If not, I would like to add this myself.

 The key problem to solve is how to not double the memory use of the
 XMLHttpRequest object, while not making responseText and responseXML slow.

 See also: https://bugs.webkit.org/show_bug.cgi?id=40954. Do we need
 both responseBody and responseArrayBuffer?

 - WBR, Alexey Proskuryakov

 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev




___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-27 Thread Anne van Kesteren
(I'm subscribed to webkit-dev with a different address.)

On Mon, Sep 27, 2010 at 8:31 PM, Michael Nordman micha...@google.com wrote:
 Yes, if we go with telling xhr up front for the array buffer case, I guess
 an enum would be slightly better.

I do not think this should be told up front. You already need to keep
the octet buffer in memory for overrideMimeType to work the way it
does. We designed responseBlob as an optimization so you would not
have to deal with the other types. I do not think you can optimize the
reverse with the design as it stands today.

In any event, any discussion on changes to the specification really
ought to happen on public-weba...@w3.org.

Cheers,


-- 
Anne van Kesteren
http://annevankesteren.nl/
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-27 Thread Michael Nordman
Webkit's XHR currently does not keep two copies of the data that I can see.
I think we should avoid that.

On Mon, Sep 27, 2010 at 2:40 PM, Anne van Kesteren 
annevankeste...@gmail.com wrote:

 (I'm subscribed to webkit-dev with a different address.)

 On Mon, Sep 27, 2010 at 8:31 PM, Michael Nordman micha...@google.com
 wrote:
  Yes, if we go with telling xhr up front for the array buffer case, I
 guess
  an enum would be slightly better.

 I do not think this should be told up front. You already need to keep
 the octet buffer in memory for overrideMimeType to work the way it
 does. We designed responseBlob as an optimization so you would not
 have to deal with the other types. I do not think you can optimize the
 reverse with the design as it stands today.

 In any event, any discussion on changes to the specification really
 ought to happen on public-weba...@w3.org.

 Cheers,


 --
 Anne van Kesteren
 http://annevankesteren.nl/

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-27 Thread Maciej Stachowiak

On Sep 27, 2010, at 3:19 PM, Michael Nordman wrote:

 Webkit's XHR currently does not keep two copies of the data that I can see. I 
 think we should avoid that.

We could keep the raw data around, which hopefully is directly usable as an 
ArrayBuffer backing store, and only translate it to text format when/if the 
client requests responseText. 

 
 On Mon, Sep 27, 2010 at 2:40 PM, Anne van Kesteren 
 annevankeste...@gmail.com wrote:
 (I'm subscribed to webkit-dev with a different address.)
 
 On Mon, Sep 27, 2010 at 8:31 PM, Michael Nordman micha...@google.com wrote:
  Yes, if we go with telling xhr up front for the array buffer case, I guess
  an enum would be slightly better.
 
 I do not think this should be told up front. You already need to keep
 the octet buffer in memory for overrideMimeType to work the way it
 does. We designed responseBlob as an optimization so you would not
 have to deal with the other types. I do not think you can optimize the
 reverse with the design as it stands today.
 
 In any event, any discussion on changes to the specification really
 ought to happen on public-weba...@w3.org.
 
 Cheers,
 
 
 --
 Anne van Kesteren
 http://annevankesteren.nl/
 
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-27 Thread James Robinson
On Mon, Sep 27, 2010 at 2:40 PM, Anne van Kesteren 
annevankeste...@gmail.com wrote:

 (I'm subscribed to webkit-dev with a different address.)

 On Mon, Sep 27, 2010 at 8:31 PM, Michael Nordman micha...@google.com
 wrote:
  Yes, if we go with telling xhr up front for the array buffer case, I
 guess
  an enum would be slightly better.

 I do not think this should be told up front. You already need to keep
 the octet buffer in memory for overrideMimeType to work the way it
 does. We designed responseBlob as an optimization so you would not
 have to deal with the other types. I do not think you can optimize the
 reverse with the design as it stands today.


I see the source of confusion.  I take it that your reading of
http://www.w3.org/TR/XMLHttpRequest2/#the-overridemimetype-method is that if
the mime type is ever overriden and there is a subsequent access of
.responseText, the response body must be re-decoded with the encoding
specified by the new mimetype.

WebKit does not  implement this functionality - the response text decoder is
initialized when the first byte is received from the network based on the
override mime type (if present) and response encoding at that time.  The
decoding behavior from this point on does not change.  WebKit does not save
the raw bytes off the network between receiving chunks of data from the
network, instead it decodes chunks as they arrive and saves them as UTF16
strings.  Frankly, being able to arbitrarily change the encoding at any
point and time and ask for the responseText using a different encoding
sounds dumb - but I'm guessing you will say that is a debate for
public-webapps@

- James


 In any event, any discussion on changes to the specification really
 ought to happen on public-weba...@w3.org.

 Cheers,


 --
 Anne van Kesteren
 http://annevankesteren.nl/
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-27 Thread James Robinson
On Mon, Sep 27, 2010 at 6:37 PM, Maciej Stachowiak m...@apple.com wrote:


 On Sep 27, 2010, at 3:19 PM, Michael Nordman wrote:

 Webkit's XHR currently does not keep two copies of the data that I can see.
 I think we should avoid that.


 We could keep the raw data around, which hopefully is directly usable as an
 ArrayBuffer backing store, and only translate it to text format when/if the
 client requests responseText.


It would be unfortunate to have to keep the raw data around after the page
access .responseText, given that the overwhelming majority of pages will
touch .responseText and nothing else.  I found when improving the V8 XHR
implementation that the memory footprint of .responseText being held off of
the XHR object itself was often significant so I would be very reluctant to
grow it by an addition 50-100% (depending on encoding) in the common case.

- James
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


[webkit-dev] XHR responseArrayBuffer attribute

2010-09-24 Thread Chris Rogers
I've noticed that the responseArrayBuffer attribute has recently been added
to the XMLHttpRequest-2 specification:

http://dev.w3.org/2006/webapi/XMLHttpRequest-2/#the-responsearraybuffer-attribute

I was interested to know if anybody was planning on implementing that
attribute soon.  If not, I would like to add this myself.

Regards,
Chris Rogers
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-24 Thread Alexey Proskuryakov

24.09.2010, в 16:37, Chris Rogers написал(а):

 I was interested to know if anybody was planning on implementing that 
 attribute soon.  If not, I would like to add this myself.

The key problem to solve is how to not double the memory use of the 
XMLHttpRequest object, while not making responseText and responseXML slow.

See also: https://bugs.webkit.org/show_bug.cgi?id=40954. Do we need both 
responseBody and responseArrayBuffer?

- WBR, Alexey Proskuryakov

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-24 Thread Eric Uhrhane
On Fri, Sep 24, 2010 at 5:09 PM, Alexey Proskuryakov a...@webkit.org wrote:

 24.09.2010, в 16:37, Chris Rogers написал(а):

 I was interested to know if anybody was planning on implementing that 
 attribute soon.  If not, I would like to add this myself.

 The key problem to solve is how to not double the memory use of the 
 XMLHttpRequest object, while not making responseText and responseXML slow.

 See also: https://bugs.webkit.org/show_bug.cgi?id=40954. Do we need both 
 responseBody and responseArrayBuffer?

responseBody is the name in the XHR2 spec as of right now, but the
discussion at 
http://lists.w3.org/Archives/Public/public-webapps/2010JulSep/1019.html
suggests that it will end up as responseArrayBuffer, so we probably
only need the latter.

Eric
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-24 Thread Michael Nordman
With xhr.responseBlob we chose to have the caller decide up front and tell
the xhr object how it would like the response by setting the xhr.asBlob
attribute prior to calling send(). We could do the same with
xhr.asArrayBuffer.

On Fri, Sep 24, 2010 at 5:09 PM, Alexey Proskuryakov a...@webkit.org wrote:


 24.09.2010, в 16:37, Chris Rogers написал(а):

  I was interested to know if anybody was planning on implementing that
 attribute soon.  If not, I would like to add this myself.

 The key problem to solve is how to not double the memory use of the
 XMLHttpRequest object, while not making responseText and responseXML slow.

 See also: https://bugs.webkit.org/show_bug.cgi?id=40954. Do we need both
 responseBody and responseArrayBuffer?

 - WBR, Alexey Proskuryakov

 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


Re: [webkit-dev] XHR responseArrayBuffer attribute

2010-09-24 Thread Chris Rogers
If we added xhr.asArrayBuffer, what would happen if xhr.asBlob was also set?
 Don't we really want something like xhr.loadAsType with different enum
values for text, blob, array buffer, etc.?

On Fri, Sep 24, 2010 at 5:19 PM, Michael Nordman micha...@google.comwrote:

 With xhr.responseBlob we chose to have the caller decide up front and tell
 the xhr object how it would like the response by setting the xhr.asBlob
 attribute prior to calling send(). We could do the same with
 xhr.asArrayBuffer.

 On Fri, Sep 24, 2010 at 5:09 PM, Alexey Proskuryakov a...@webkit.orgwrote:


 24.09.2010, в 16:37, Chris Rogers написал(а):

  I was interested to know if anybody was planning on implementing that
 attribute soon.  If not, I would like to add this myself.

 The key problem to solve is how to not double the memory use of the
 XMLHttpRequest object, while not making responseText and responseXML slow.

 See also: https://bugs.webkit.org/show_bug.cgi?id=40954. Do we need
 both responseBody and responseArrayBuffer?

 - WBR, Alexey Proskuryakov

 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev



___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev