Re: [XHR] chunked
On Tue, Oct 2, 2012 at 2:56 PM, Adrian Bateman adria...@microsoft.com wrote: On Thursday, September 27, 2012 10:43 AM, Travis Leithead wrote: In my observation of the current IE behavior, the Stream is for download only. XHR gets the data from the server and buffers it. The consumer of the stream then pulls data as needed which is extracted from the buffer. In IE10, we only implemented the download part of our proposal. The idea is that you should be able to upload a continuing stream using the stream builder API. However, not many services support chunked upload nor does our underlying network stack on the client so it was a low priority and not something we've tackled yet. Just to be clear here, initially I thought http://dvcs.w3.org/hg/streams-api/raw-file/tip/Overview.htm#send0 to be too unclear for the streaming scenario, but I suppose it can work if you assume the raw data by the stream object is not a constant until the object is closed. I will take a stab at updating http://xhr.spec.whatwg.org/ with the relevant text and references. -- http://annevankesteren.nl/
Re: [XHR] chunked
On Thu, Oct 11, 2012 at 1:25 PM, Anne van Kesteren ann...@annevk.nl wrote: [...] Thanks all for your patience. http://xhr.spec.whatwg.org/ now includes the Streams API. https://github.com/whatwg/xhr/commit/4b56e0f353362d50b87539ff906519892ee13652 documents the change. -- http://annevankesteren.nl/
Re: [XHR] chunked
On Thu, Jun 7, 2012 at 11:48 AM, Wenbo Zhu wen...@google.com wrote: On Thu, Jun 7, 2012 at 2:34 AM, Anne van Kesteren ann...@annevk.nl wrote: Can I take this as Chrome being interested in implementing the chunked proposal as well? Will have to get back to you on this, since I am mostly working on servers and HTTP/application protocols. Still trying to figure out what to do here. Any other XMLHttpRequest implementors hoping to implement this feature? Has it been adopted by developers? -- http://annevankesteren.nl/
Re: [XHR] chunked
On Thu, Sep 27, 2012 at 6:27 AM, Anne van Kesteren ann...@annevk.nl wrote: On Thu, Jun 7, 2012 at 11:48 AM, Wenbo Zhu wen...@google.com wrote: On Thu, Jun 7, 2012 at 2:34 AM, Anne van Kesteren ann...@annevk.nl wrote: Can I take this as Chrome being interested in implementing the chunked proposal as well? Will have to get back to you on this, since I am mostly working on servers and HTTP/application protocols. Still trying to figure out what to do here. Any other XMLHttpRequest implementors hoping to implement this feature? Has it been adopted by developers? I know pdf.js uses this in order to implement incremental rendering of pdf files. I do somewhat agree that if we had a full stream solution in the form of a Stream primitive and .responseType=stream, then a better solution might be to use that in combination with having chunked delivery on the Stream class instead. / Jonas
Re: [XHR] chunked
On Thu, Sep 27, 2012 at 6:23 PM, Jonas Sicking jo...@sicking.cc wrote: I do somewhat agree that if we had a full stream solution in the form of a Stream primitive and .responseType=stream, then a better solution might be to use that in combination with having chunked delivery on the Stream class instead. Was Microsoft going to take a stab at that? I could write something like that up in XMLHttpRequest I suppose. Although maybe that would require a bunch of coordination with the MediaStream stuff? -- http://annevankesteren.nl/
Re: [XHR] chunked
On Thu, Sep 27, 2012 at 9:26 AM, Anne van Kesteren ann...@annevk.nl wrote: On Thu, Sep 27, 2012 at 6:23 PM, Jonas Sicking jo...@sicking.cc wrote: I do somewhat agree that if we had a full stream solution in the form of a Stream primitive and .responseType=stream, then a better solution might be to use that in combination with having chunked delivery on the Stream class instead. Was Microsoft going to take a stab at that? I could write something like that up in XMLHttpRequest I suppose. Although maybe that would require a bunch of coordination with the MediaStream stuff? I can't speak to anyone else's plans. But it does seem like a proposal was made quite a long time ago and as I recall it it received favorable feedback, but so far nothing else has happened. / Jonas
RE: [XHR] chunked
From: annevankeste...@gmail.com [mailto:annevankeste...@gmail.com] On On Thu, Sep 27, 2012 at 7:00 PM, Travis Leithead travis.leith...@microsoft.com wrote: It hasn't been updated in a while, but we're still keen on seeing it move forward AFAIK: http://dvcs.w3.org/hg/streams-api/raw-file/tip/Overview.htm Cool. I can integrate the relevant bits into XMLHttpRequest. Do I understand it correctly that it does not allow for streaming data towards the server? It seems to just copy the data from the Stream object that it currently represents and that's that. Is that really what we want here? In my observation of the current IE behavior, the Stream is for download only. XHR gets the data from the server and buffers it. The consumer of the stream then pulls data as needed which is extracted from the buffer.
Re: [XHR] chunked
On Thu, Sep 27, 2012 at 7:42 PM, Travis Leithead travis.leith...@microsoft.com wrote: In my observation of the current IE behavior, the Stream is for download only. XHR gets the data from the server and buffers it. The consumer of the stream then pulls data as needed which is extracted from the buffer. I see, so the bit where it says you can pass it to send() we should maybe not add for now if it's not going to do something useful. -- http://annevankesteren.nl/
RE: [XHR] chunked
From: annevankeste...@gmail.com [mailto:annevankeste...@gmail.com] On Thu, Sep 27, 2012 at 7:42 PM, Travis Leithead travis.leith...@microsoft.com wrote: In my observation of the current IE behavior, the Stream is for download only. XHR gets the data from the server and buffers it. The consumer of the stream then pulls data as needed which is extracted from the buffer. I see, so the bit where it says you can pass it to send() we should maybe not add for now if it's not going to do something useful. I honestly haven't tested that part, but this seems safe (it can be added- in later if need be).
Re: [XHR] chunked
On Thu, Sep 27, 2012 at 12:21 PM, Travis Leithead travis.leith...@microsoft.com wrote: From: annevankeste...@gmail.com [mailto:annevankeste...@gmail.com] On Thu, Sep 27, 2012 at 7:42 PM, Travis Leithead travis.leith...@microsoft.com wrote: In my observation of the current IE behavior, the Stream is for download only. XHR gets the data from the server and buffers it. The consumer of the stream then pulls data as needed which is extracted from the buffer. I see, so the bit where it says you can pass it to send() we should maybe not add for now if it's not going to do something useful. I honestly haven't tested that part, but this seems safe (it can be added- in later if need be). The send() version, i.e. chunked requests, will actually be more useful, but I guess someone has to use it first, e.g. to cut the number of requests.
Re: [XHR] chunked
On Thu, May 24, 2012 at 8:59 AM, Anne van Kesteren ann...@annevk.nl wrote: On Thu, May 24, 2012 at 2:54 AM, Jonas Sicking jo...@sicking.cc wrote: Is there a reason not to add chunked-text and chunked-arraybuffer to the spec right now? 1. Why not just have http://html5labs.interoperabilitybridges.com/streamsapi/ to address this case? It appears that Microsoft's proposal involves potentially buffering the XHR response body until the Web author-supplied script chooses to initiate I read from the stream object. Or am I missing something? The chunked-text and the chunked-arraybuffer response types are both simpler than the stream proposal and have the advantage that they don't involve the browser engine buffering the XHR response body: if the Web author-provided script fails to handle the chunks as they arrive, the data is gone and doesn't fill up buffer space. (Some of my remarks on IRC were confused, because I mistook the Worker-specific API for the API proposed for the main thread.) -- Henri Sivonen hsivo...@iki.fi http://hsivonen.iki.fi/
Re: [XHR] chunked
On Thu, Jun 7, 2012 at 2:14 AM, Henri Sivonen hsivo...@iki.fi wrote: On Thu, May 24, 2012 at 8:59 AM, Anne van Kesteren ann...@annevk.nl wrote: On Thu, May 24, 2012 at 2:54 AM, Jonas Sicking jo...@sicking.cc wrote: Is there a reason not to add chunked-text and chunked-arraybuffer to the spec right now? 1. Why not just have http://html5labs.interoperabilitybridges.com/streamsapi/ to address this case? It appears that Microsoft's proposal involves potentially buffering the XHR response body until the Web author-supplied script chooses to initiate I read from the stream object. Or am I missing something? The chunked-text and the chunked-arraybuffer response types are both simpler than the stream proposal and have the advantage that they don't involve the browser engine buffering the XHR response body: if the Web author-provided script fails to handle the chunks as they arrive, the data is gone and doesn't fill up buffer space. Yes, this will be the desired behavior. (Some of my remarks on IRC were confused, because I mistook the Worker-specific API for the API proposed for the main thread.) -- Henri Sivonen hsivo...@iki.fi http://hsivonen.iki.fi/
Re: [XHR] chunked
On Thu, Jun 7, 2012 at 11:30 AM, Wenbo Zhu wen...@google.com wrote: Yes, this will be the desired behavior. Can I take this as Chrome being interested in implementing the chunked proposal as well? Because if only Mozilla is interested in these I don't see much point adding them to the standard, as I indicated before. -- Anne — Opera Software http://annevankesteren.nl/ http://www.opera.com/
Re: [XHR] chunked
On Thu, Jun 7, 2012 at 2:34 AM, Anne van Kesteren ann...@annevk.nl wrote: On Thu, Jun 7, 2012 at 11:30 AM, Wenbo Zhu wen...@google.com wrote: Yes, this will be the desired behavior. Can I take this as Chrome being interested in implementing the chunked proposal as well? Will have to get back to you on this, since I am mostly working on servers and HTTP/application protocols. Because if only Mozilla is interested in these I don't see much point adding them to the standard, as I indicated before. I suppose you still favor the idea of exposing chunked HTTP to applications. Without it, native clients will be increasingly deployed in place of HTML5 apps. -- Anne — Opera Software http://annevankesteren.nl/ http://www.opera.com/
Re: [XHR] chunked
On Thu, Jun 7, 2012 at 11:48 AM, Wenbo Zhu wen...@google.com wrote: I suppose you still favor the idea of exposing chunked HTTP to applications. Without it, native clients will be increasingly deployed in place of HTML5 apps. Lets not play scare-tactics. And besides, there's server-sent events and the WebSocket protocol. -- Anne — Opera Software http://annevankesteren.nl/ http://www.opera.com/
Re: [XHR] chunked
On Thu, May 24, 2012 at 2:54 AM, Jonas Sicking jo...@sicking.cc wrote: No change came out of this thread. I think we established that there is a need for chunked-arraybuffer. This thread contained some controversy about whether chunked-text is needed, but in the end the controversy didn't actually seem relevant to the question at hand. Is there a reason not to add chunked-text and chunked-arraybuffer to the spec right now? 1. Why not just have http://html5labs.interoperabilitybridges.com/streamsapi/ to address this case? 2. Is anyone else planning on implementing these? -- Anne — Opera Software http://annevankesteren.nl/ http://www.opera.com/
Re: [XHR] chunked requests
On Sat, Dec 10, 2011 at 2:14 AM, Anne van Kesteren ann...@opera.com wrote: On Thu, 08 Dec 2011 23:16:37 +0100, Jonas Sicking jo...@sicking.cc wrote: I think Microsoft's stream proposal would address this use case. So that would be: http://html5labs.interoperabilitybridges.com/streamsapi/ How does that relate to the various APIs for streaming media? They're entirely unrelated (except for an unfortunate naming similarity). Microsoft's Streams are a very different abstraction to MediaStreams. Rob -- If we claim to be without sin, we deceive ourselves and the truth is not in us. If we confess our sins, he is faithful and just and will forgive us our sins and purify us from all unrighteousness. If we claim we have not sinned, we make him out to be a liar and his word is not in us. [1 John 1:8-10]
Re: [XHR] chunked requests
On Sun, 18 Dec 2011 13:12:57 +0100, Eric Rescorla e...@rtfm.com wrote: Sorry, I forgot to mention the 1/n+1 splitting countermeasure in my response. With that said, this isn't TLS 1.1, but rather a specific, more backwards-compatible countermeasure. It's fine for the security considerations section to say here that browsers must do either TLS 1.1 or 1/n+1 splitting, but it should say something, since it's not like 1/n+1 splitting is required by TLS (any version). Who's in charge of updating TLS? Surely this should be patched in the base specification rather than in every API that interacts with it. I do not want to make the life of the guy implementing XMLHttpRequest more difficult if the problem is supposed to be addressed at the TLS layer anyway. -- Anne van Kesteren http://annevankesteren.nl/
Re: [XHR] chunked requests
On Tue, Dec 20, 2011 at 9:36 AM, Anne van Kesteren ann...@opera.com wrote: On Sun, 18 Dec 2011 13:12:57 +0100, Eric Rescorla e...@rtfm.com wrote: Sorry, I forgot to mention the 1/n+1 splitting countermeasure in my response. With that said, this isn't TLS 1.1, but rather a specific, more backwards-compatible countermeasure. It's fine for the security considerations section to say here that browsers must do either TLS 1.1 or 1/n+1 splitting, but it should say something, since it's not like 1/n+1 splitting is required by TLS (any version). Who's in charge of updating TLS? Me. Surely this should be patched in the base specification rather than in every API that interacts with it. I do not want to make the life of the guy implementing XMLHttpRequest more difficult if the problem is supposed to be addressed at the TLS layer anyway. The problem was addressed at the TLS layer 5 years ago when we issued TLS 1.1. -Ekr
Re: [XHR] chunked requests
On Tue, 20 Dec 2011 21:06:28 +0100, Eric Rescorla e...@rtfm.com wrote: On Tue, Dec 20, 2011 at 9:36 AM, Anne van Kesteren ann...@opera.com wrote: Surely this should be patched in the base specification rather than in every API that interacts with it. I do not want to make the life of the guy implementing XMLHttpRequest more difficult if the problem is supposed to be addressed at the TLS layer anyway. The problem was addressed at the TLS layer 5 years ago when we issued TLS 1.1. If support for the old version cannot be removed it does not seem to me that the problem is actually addressed as far as user agents that have to support legacy servers are concerned. -- Anne van Kesteren http://annevankesteren.nl/
Re: [XHR] chunked requests
On Tue, Dec 20, 2011 at 12:11 PM, Anne van Kesteren ann...@opera.com wrote: On Tue, 20 Dec 2011 21:06:28 +0100, Eric Rescorla e...@rtfm.com wrote: On Tue, Dec 20, 2011 at 9:36 AM, Anne van Kesteren ann...@opera.com wrote: Surely this should be patched in the base specification rather than in every API that interacts with it. I do not want to make the life of the guy implementing XMLHttpRequest more difficult if the problem is supposed to be addressed at the TLS layer anyway. The problem was addressed at the TLS layer 5 years ago when we issued TLS 1.1. If support for the old version cannot be removed it does not seem to me that the problem is actually addressed as far as user agents that have to support legacy servers are concerned. I see your point, but it's not clear what you would like done about it. The TLS WG specifies protocols, not APIs, and this is an API issue. Even if it were within scope, all we would do is add some sort of security considerations note to the TLS spec, but again, it would only apply to RFC 2246, since TLS 1.1 and 1.2 don't have this problem. That isn't to say that the browser stacks aren't adding 1/n+1 splitting. NSS, for instance, has such a fix. However, I don't think there's anything to do from a TLS standards perspective. -Ekr
Re: [XHR] chunked requests
On Tue, 20 Dec 2011 22:55:40 +0100, Eric Rescorla e...@rtfm.com wrote: That isn't to say that the browser stacks aren't adding 1/n+1 splitting. NSS, for instance, has such a fix. However, I don't think there's anything to do from a TLS standards perspective. What I would like is a standard that actually defines what of TLS user agents need to implement. Instead of every user agent having to figure that out for themselves. Those unknowns (TLS is not alone here) create a barrier to entry and make it harder to create user agents that interoperate with legacy content. -- Anne van Kesteren http://annevankesteren.nl/
Re: [XHR] chunked requests
On Tue, Dec 20, 2011 at 2:47 PM, Anne van Kesteren ann...@opera.com wrote: On Tue, 20 Dec 2011 22:55:40 +0100, Eric Rescorla e...@rtfm.com wrote: That isn't to say that the browser stacks aren't adding 1/n+1 splitting. NSS, for instance, has such a fix. However, I don't think there's anything to do from a TLS standards perspective. What I would like is a standard that actually defines what of TLS user agents need to implement. Instead of every user agent having to figure that out for themselves. Those unknowns (TLS is not alone here) create a barrier to entry and make it harder to create user agents that interoperate with legacy content. Again, the TLS spec does tell you what to do: it tells you to implement TLS 1.1 and/or 1.2. What you're asking for is for it *also* to tell you what to do in cases where you choose to use a downrev version of TLS. But unfortunately, that guidance is necessarily application specific. In particular, the use of 1/n+1 splitting is the result of discovering that some HTTP stacks don't properly handle empty TLS records and even then some stacks choke Moreover, many non-HTTPS stacks have no need to do any sort of countermeasure because Rizzo-Duong-style attacks don't apply. If/when IETF publishes a revision of TLS, I suspect we'll add a description of 1/n+1 splitting as a backward compatibility measure, but I don't really think it's the role of standards to document security hacks for downrev versions in perpetuity. -Ekr
Re: [XHR] chunked requests
On Sat, Dec 17, 2011 at 6:11 AM, Anne van Kesteren ann...@opera.com wrote: On Fri, 09 Dec 2011 19:54:31 +0100, Eric Rescorla e...@rtfm.com wrote: Unfortunately, many servers do not support TLS 1.1, and to make matters worse, they do so in a way that is not securely verifiable. By which I mean that an active attacker can force a client/server pair both of which support TLS 1.1 down to TLS 1.0. This may be detectable in some way, but not by TLS's built-in mechanisms. And since the threat model here is an active attacker, this is a problem. It seems user agents are addressing this issue in general by simply removing support for those servers so we might not have to define anything here and just leave it to the TLS standards: http://my.opera.com/securitygroup/blog/2011/12/11/opera-11-60-and-new-problems-with-some-secure-servers Sorry, I forgot to mention the 1/n+1 splitting countermeasure in my response. With that said, this isn't TLS 1.1, but rather a specific, more backwards-compatible countermeasure. It's fine for the security considerations section to say here that browsers must do either TLS 1.1 or 1/n+1 splitting, but it should say something, since it's not like 1/n+1 splitting is required by TLS (any version). -Ekr
Re: [XHR] chunked requests
On Fri, 09 Dec 2011 19:54:31 +0100, Eric Rescorla e...@rtfm.com wrote: Unfortunately, many servers do not support TLS 1.1, and to make matters worse, they do so in a way that is not securely verifiable. By which I mean that an active attacker can force a client/server pair both of which support TLS 1.1 down to TLS 1.0. This may be detectable in some way, but not by TLS's built-in mechanisms. And since the threat model here is an active attacker, this is a problem. It seems user agents are addressing this issue in general by simply removing support for those servers so we might not have to define anything here and just leave it to the TLS standards: http://my.opera.com/securitygroup/blog/2011/12/11/opera-11-60-and-new-problems-with-some-secure-servers -- Anne van Kesteren http://annevankesteren.nl/
Re: [XHR] chunked requests
On Sat, Dec 17, 2011 at 6:11 AM, Anne van Kesteren ann...@opera.com wrote: On Fri, 09 Dec 2011 19:54:31 +0100, Eric Rescorla e...@rtfm.com wrote: Unfortunately, many servers do not support TLS 1.1, and to make matters worse, they do so in a way that is not securely verifiable. By which I mean that an active attacker can force a client/server pair both of which support TLS 1.1 down to TLS 1.0. This may be detectable in some way, but not by TLS's built-in mechanisms. And since the threat model here is an active attacker, this is a problem. It seems user agents are addressing this issue in general by simply removing support for those servers so we might not have to define anything here and just leave it to the TLS standards: http://my.opera.com/securitygroup/blog/2011/12/11/opera-11-60-and-new-problems-with-some-secure-servers I would still add a security consideration so folks who implement this are aware that the two issues are related. Adam
RE: [XHR] chunked requests
We've only recently uploaded the draft for the Streams API and shared it with the Working Group [1], which also has the new location. There is a conversation currently taking place in the media capture WG [2] discussing how to better align Stream and MediaStream. I think it's an interesting topic worth investigating, and there are a few ideas that have been proposed. We'll have to see where the conversation lands. Thanks, Feras [1] http://lists.w3.org/Archives/Public/public-webapps/2011OctDec/1494.html [2] http://lists.w3.org/Archives/Public/public-media-capture/2011Dec/0037.html -Original Message- From: Anne van Kesteren [mailto:ann...@opera.com] Sent: Friday, December 09, 2011 5:15 AM To: Wenbo Zhu; Jonas Sicking; Robert O'Callahan; Feras Moussa Cc: WebApps WG Subject: Re: [XHR] chunked requests On Thu, 08 Dec 2011 23:16:37 +0100, Jonas Sicking jo...@sicking.cc wrote: I think Microsoft's stream proposal would address this use case. So that would be: http://html5labs.interoperabilitybridges.com/streamsapi/ How does that relate to the various APIs for streaming media? (I added roc and Feras Moussa.) -- Anne van Kesteren http://annevankesteren.nl/
Re: [XHR] chunked requests
On Wed, 14 Dec 2011 20:17:01 +0100, Feras Moussa fer...@microsoft.com wrote: We've only recently uploaded the draft for the Streams API and shared it with the Working Group [1], which also has the new location. There is a conversation currently taking place in the media capture WG [2] discussing how to better align Stream and MediaStream. I think it's an interesting topic worth investigating, and there are a few ideas that have been proposed. We'll have to see where the conversation lands. Cool! Let me know when I should start adding the relevant text to XMLHttpRequest. -- Anne van Kesteren http://annevankesteren.nl/
Re: [XHR] chunked requests
On Thu, Dec 8, 2011 at 2:36 PM, Charles Pritchard ch...@jumis.com wrote: On Dec 8, 2011, at 1:04 PM, Wenbo Zhu wen...@google.com wrote: On Wed, Dec 7, 2011 at 6:04 PM, Charles Pritchard ch...@jumis.com ch...@jumis.com wrote: ** I think the Web Sockets spec is intended for client to server sessions like this. WebSocket protocol isn't yet well-supported by proxies. Besides, all we need here is a throwaway RPC-like request/response, and it's a bit heavy duty to use WebSocket for this use case. Isn't a chunked stream the anti-thesis of RPC? Streaming request or response does not imply bi-directional messaging, i.e. the (non-RPC) semantics of WebSocket. It looks like you're acknowledging WebSockets works while asking about what to do with current web infrastructure. It seems to me that current infrastructure is -mostly- Flash based and assumes a level of control over input: one can't send arbitrary information as a developer.. It has to be directly from a webcam or similar media stream. Do you have specific examples of existing infrastructure you're hoping to support? See below. As for the peer-to-peer communication, I imagine the WebRTC group is where you'll see more activity on this issue. WebRTC is mostly peer-to-peer UDP (e.g. RTP), and the client-to-client communication has nothing to do with either HTTP or WebSocket. And MediaStream/Stream, which has to so with your request. Not sure about the status of this spec. My original email merely tries to show the need to process/post data locally ... http://developers.whatwg.org/video-conferencing-and-peer-to-peer-communication.html#video-conferencing-and-peer-to-peer-communication I don't know that arbitrary binary data is on their agenda -- I hope it is -- for p2p communication. On 12/7/11 5:59 PM, Wenbo Zhu wrote: One use case that we have which is not currently handled by XMLHttpRequest is incrementally sending data that takes a long time to generate _from the client to the server_. For example, if we were to record data from a microphone, we couldn't upload it in real time to the server with the current API. The MediaStreaming spec also mentioned several use cases which would require streaming request data via an API: - Sending the locally-produced streams to remote peers and receiving streams from remote peers. - Sending arbitrary data to remote peers. http://www.whatwg.org/specs/web-apps/current-work/multipage/video-conferencing-and-peer-to-peer-communication.html - Wenbo
Re: [XHR] chunked requests
On Fri, 09 Dec 2011 02:13:50 +0100, Eric Rescorla e...@rtfm.com wrote: On Thu, Dec 8, 2011 at 5:07 PM, Adam Barth w...@adambarth.com wrote: Whatever spec we end up going with should note in its security consideration that the user agent must implement TLS 1.2 or greater to avoid this attack. I believe it's actually TLS 1.1, since the relevant feature is explicit IVs. Or you could allow RC4, I guess. Are you saying that if responseType is set to stream and the server only supports TLS 1.0 the connection should fail, but if it is greater than that it is okay? Same-origin requests are always okay? (Though it seems we should just require TLS 1.1 there too then to not make matters too confusing.) -- Anne van Kesteren http://annevankesteren.nl/
Re: [XHR] chunked requests
On Thu, 08 Dec 2011 23:16:37 +0100, Jonas Sicking jo...@sicking.cc wrote: I think Microsoft's stream proposal would address this use case. So that would be: http://html5labs.interoperabilitybridges.com/streamsapi/ How does that relate to the various APIs for streaming media? (I added roc and Feras Moussa.) -- Anne van Kesteren http://annevankesteren.nl/
Re: [XHR] chunked requests
On Fri, Dec 9, 2011 at 4:59 AM, Anne van Kesteren ann...@opera.com wrote: On Fri, 09 Dec 2011 02:13:50 +0100, Eric Rescorla e...@rtfm.com wrote: On Thu, Dec 8, 2011 at 5:07 PM, Adam Barth w...@adambarth.com wrote: Whatever spec we end up going with should note in its security consideration that the user agent must implement TLS 1.2 or greater to avoid this attack. I believe it's actually TLS 1.1, since the relevant feature is explicit IVs. Or you could allow RC4, I guess. Are you saying that if responseType is set to stream and the server only supports TLS 1.0 the connection should fail, but if it is greater than that it is okay? I'm not sure I understand this feature well enough. The issue is streaming content from the client, not from the server, and in particular the ability of the JS to provide additional content to be sent after the data has started to be transmitted. As for what should happen in this setting if the negotiated TLS version is 1.0, I could imagine a number of possibilities, including: 1. The client refuses to send 2. There is a pre-flight and the server has to explicitly accept. 3. There is a big nasty warning. 4. We just warn people in the spec and hope they do something sensible Same-origin requests are always okay? (Though it seems we should just require TLS 1.1 there too then to not make matters too confusing.) Same-origin requests should be OK because the JS would have access to the relevant sensitive data in any case. -Ekr
Re: [XHR] chunked requests
On Fri, 09 Dec 2011 16:33:08 +0100, Eric Rescorla e...@rtfm.com wrote: On Fri, Dec 9, 2011 at 4:59 AM, Anne van Kesteren ann...@opera.com wrote: Are you saying that if responseType is set to stream and the server only supports TLS 1.0 the connection should fail, but if it is greater than that it is okay? I'm not sure I understand this feature well enough. The issue is streaming content from the client, not from the server, and in particular the ability of the JS to provide additional content to be sent after the data has started to be transmitted. My bad. I meant send(Stream) which would indeed allow for that. As for what should happen in this setting if the negotiated TLS version is 1.0, I could imagine a number of possibilities, including: 1. The client refuses to send 2. There is a pre-flight and the server has to explicitly accept. 3. There is a big nasty warning. 4. We just warn people in the spec and hope they do something sensible Okay. I think I would very much prefer 1 here. Same-origin requests are always okay? (Though it seems we should just require TLS 1.1 there too then to not make matters too confusing.) Same-origin requests should be OK because the JS would have access to the relevant sensitive data in any case. Okay, I guess we can make that difference. Thanks, -- Anne van Kesteren http://annevankesteren.nl/
Re: [XHR] chunked requests
On Fri, Dec 9, 2011 at 7:59 AM, Anne van Kesteren ann...@opera.com wrote: On Fri, 09 Dec 2011 16:33:08 +0100, Eric Rescorla e...@rtfm.com wrote: On Fri, Dec 9, 2011 at 4:59 AM, Anne van Kesteren ann...@opera.com wrote: Are you saying that if responseType is set to stream and the server only supports TLS 1.0 the connection should fail, but if it is greater than that it is okay? I'm not sure I understand this feature well enough. The issue is streaming content from the client, not from the server, and in particular the ability of the JS to provide additional content to be sent after the data has started to be transmitted. My bad. I meant send(Stream) which would indeed allow for that. As for what should happen in this setting if the negotiated TLS version is 1.0, I could imagine a number of possibilities, including: 1. The client refuses to send 2. There is a pre-flight and the server has to explicitly accept. 3. There is a big nasty warning. 4. We just warn people in the spec and hope they do something sensible Okay. I think I would very much prefer 1 here. Same-origin requests are always okay? (Though it seems we should just require TLS 1.1 there too then to not make matters too confusing.) Same-origin requests should be OK because the JS would have access to the relevant sensitive data in any case. Okay, I guess we can make that difference. Correct me if I'm wrong, but I believe these issues are fixed in TLS 1.1. Most user agents implement TLS 1.1 anyway, so this seems mostly like a requirement to put in the security considerations section. Adam
Re: [XHR] chunked requests
On Fri, Dec 9, 2011 at 10:37 AM, Adam Barth w...@adambarth.com wrote: On Fri, Dec 9, 2011 at 7:59 AM, Anne van Kesteren ann...@opera.com wrote: On Fri, 09 Dec 2011 16:33:08 +0100, Eric Rescorla e...@rtfm.com wrote: Same-origin requests should be OK because the JS would have access to the relevant sensitive data in any case. Okay, I guess we can make that difference. Correct me if I'm wrong, but I believe these issues are fixed in TLS 1.1. Most user agents implement TLS 1.1 anyway, so this seems mostly like a requirement to put in the security considerations section. Would that it were this easy. Unfortunately, many servers do not support TLS 1.1, and to make matters worse, they do so in a way that is not securely verifiable. By which I mean that an active attacker can force a client/server pair both of which support TLS 1.1 down to TLS 1.0. This may be detectable in some way, but not by TLS's built-in mechanisms. And since the threat model here is an active attacker, this is a problem. -Ekr
Re: [XHR] chunked requests
On Dec 8, 2011, at 1:04 PM, Wenbo Zhu wen...@google.com wrote: On Wed, Dec 7, 2011 at 6:04 PM, Charles Pritchard ch...@jumis.com wrote: I think the Web Sockets spec is intended for client to server sessions like this. WebSocket protocol isn't yet well-supported by proxies. Besides, all we need here is a throwaway RPC-like request/response, and it's a bit heavy duty to use WebSocket for this use case. Isn't a chunked stream the anti-thesis of RPC? It looks like you're acknowledging WebSockets works while asking about what to do with current web infrastructure. It seems to me that current infrastructure is -mostly- Flash based and assumes a level of control over input: one can't send arbitrary information as a developer.. It has to be directly from a webcam or similar media stream. Do you have specific examples of existing infrastructure you're hoping to support? As for the peer-to-peer communication, I imagine the WebRTC group is where you'll see more activity on this issue. WebRTC is mostly peer-to-peer UDP (e.g. RTP), and the client-to-client communication has nothing to do with either HTTP or WebSocket. And MediaStream/Stream, which has to so with your request. I don't know that arbitrary binary data is on their agenda -- I hope it is -- for p2p communication. On 12/7/11 5:59 PM, Wenbo Zhu wrote: One use case that we have which is not currently handled by XMLHttpRequest is incrementally sending data that takes a long time to generate _from the client to the server_. For example, if we were to record data from a microphone, we couldn't upload it in real time to the server with the current API. The MediaStreaming spec also mentioned several use cases which would require streaming request data via an API: - Sending the locally-produced streams to remote peers and receiving streams from remote peers. - Sending arbitrary data to remote peers. http://www.whatwg.org/specs/web-apps/current-work/multipage/video-conferencing-and-peer-to-peer-communication.html - Wenbo
Re: [XHR] chunked requests
Keep in mind that streamed or chunked uploads will expose the ability to exploit the BEAST vulnerability in SSL: http://www.educatedguesswork.org/2011/09/security_impact_of_the_rizzodu.html Whatever spec we end up going with should note in its security consideration that the user agent must implement TLS 1.2 or greater to avoid this attack. Adam On Thu, Dec 8, 2011 at 2:16 PM, Jonas Sicking jo...@sicking.cc wrote: I think Microsoft's stream proposal would address this use case. / Jonas On Wed, Dec 7, 2011 at 5:59 PM, Wenbo Zhu wen...@google.com wrote: One use case that we have which is not currently handled by XMLHttpRequest is incrementally sending data that takes a long time to generate _from the client to the server_. For example, if we were to record data from a microphone, we couldn't upload it in real time to the server with the current API. The MediaStreaming spec also mentioned several use cases which would require streaming request data via an API: - Sending the locally-produced streams to remote peers and receiving streams from remote peers. - Sending arbitrary data to remote peers. http://www.whatwg.org/specs/web-apps/current-work/multipage/video-conferencing-and-peer-to-peer-communication.html - Wenbo
Re: [XHR] chunked requests
On Thu, Dec 8, 2011 at 5:07 PM, Adam Barth w...@adambarth.com wrote: Keep in mind that streamed or chunked uploads will expose the ability to exploit the BEAST vulnerability in SSL: http://www.educatedguesswork.org/2011/09/security_impact_of_the_rizzodu.html Right. Specifically, it needs to be a cross-origin streamed request without significant uncontrollable headers and/or masking. Whatever spec we end up going with should note in its security consideration that the user agent must implement TLS 1.2 or greater to avoid this attack. I believe it's actually TLS 1.1, since the relevant feature is explicit IVs. Or you could allow RC4, I guess. Best, -Ekr
Re: [XHR] chunked
On Wed, Nov 30, 2011 at 7:28 AM, Anne van Kesteren ann...@opera.com wrote: A while ago sicking proposed adding chunked support to XMLHttpRequest: http://lists.w3.org/Archives/**Public/public-webapps/** 2011JulSep/0741.htmlhttp://lists.w3.org/Archives/Public/public-webapps/2011JulSep/0741.html https://bugzilla.mozilla.org/**show_bug.cgi?id=687087https://bugzilla.mozilla.org/show_bug.cgi?id=687087 A use case I remember was downloading a large file of some kind that presumably can be incrementally rendered as otherwise responseType blob should be sufficient. More use cases appreciated. Would help with the design. E.g. voice/image search, translation ... IMHO any single resource that involves non-trivial processing to produce would fit the use case. As for the feature, basically have responseType chunked-text and chunked-arraybuffer values and reset rather than update the response entity body with each progress event. And make sure that a progress event is dispatched when the last fetch event is queued. And make sure that this is only available for asynchronous usage. Charles asked whether chunked-text was really needed (and whether we should have chunked which implies ArrayBuffer instead). Nobody got back to him on that. If it is needed, how does it work when you just have some of the bytes of a multi-byte character in a single chunk? Fails to decode as per the normal algorithm? When text is consumed as chunked streams, my take is that the application code has to deal with partial frames, and partial chars are just one sub-problem. So, I wouldn't consider multi-byte characters a particular limitation. Also, this basically makes it possible to write EventSource on top of XMLHttpRequest. Is that acceptable? If it encourages more people to use a lower-level API, higher-level optimizations for mobile phones might become harder down the road. At the same time, lower-level APIs that match the underlying wire-protocol (i.e. HTTP) would be equally important for optimization purposes. Thanks, Wenbo -- Anne van Kesteren http://annevankesteren.nl/
Re: [XHR] chunked
On Wed, Nov 30, 2011 at 7:28 AM, Anne van Kesteren ann...@opera.com wrote: A while ago sicking proposed adding chunked support to XMLHttpRequest: http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/0741.html https://bugzilla.mozilla.org/show_bug.cgi?id=687087 A use case I remember was downloading a large file of some kind that presumably can be incrementally rendered as otherwise responseType blob should be sufficient. More use cases appreciated. Would help with the design. Incremental rendering as well as incremental parsing. Imagine for example a webapp like google docs but with the ability to read MS-Word documents directly client-side (currently I believe google docs does their MS-Word file parsing server-side). Even if the application wouldn't want to render the document right away, parsing it incrementally rather than after the full download has several advantages. First of all, external resources referenced from the document can be loaded immediately and in parallel with the main document. Second not having to keep the unparsed data of the full document can save a bunch of memory. Third, it means that the document is available for use sooner after the download is finished since most if it is already parsed. Charles asked whether chunked-text was really needed (and whether we should have chunked which implies ArrayBuffer instead). Nobody got back to him on that. Any text based format would benefit from chunked-text. While the example above uses a binary format, it applies equally to text based formats. And given how much we in this group seem to prefer text based formats, (HTML, CSS, Javascript, EventSource, JSON) I think we should assume that other people at least use them, if not prefer them. If it is needed, how does it work when you just have some of the bytes of a multi-byte character in a single chunk? Fails to decode as per the normal algorithm? Any multi-byte characters which are only partially downloaded are not returned in that chunk. I.e. you only expose fully decoded characters. The value in progressEvent.loaded doesn't match up with the accumulated xhr.response.length for multi-byte encodings anyway. Also, this basically makes it possible to write EventSource on top of XMLHttpRequest. Is that acceptable? If it encourages more people to use a lower-level API, higher-level optimizations for mobile phones might become harder down the road. It seems more than acceptable to me! It's a good sign if our low-level primitives, like XHR, are extended to the point that they can be used to implement our higher level APIs. That shows that we're building our low-level primitives correctly. And note that EventSource is already implementable on top of XHR. The only thing that chunked is buying us is less memory usage. But I also think that EventSource still has some significant advantages in that it supplies things like automatic reconnection and possibly in the future events delivered over SMS or other push mechanisms. / Jonas
Re: [XHR] chunked
On Nov 30, 2011, at 11:32 AM, Jonas Sicking jo...@sicking.cc wrote: Charles asked whether chunked-text was really needed (and whether we should have chunked which implies ArrayBuffer instead). Nobody got back to him on that. Any text based format would benefit from chunked-text. While the example above uses a binary format, it applies equally to text based formats. And given how much we in this group seem to prefer text based formats, (HTML, CSS, Javascript, EventSource, JSON) I think we should assume that other people at least use them, if not prefer them. My thinking was that ArrayBuffer can easily be converted to String by authors. Even with text-based formats, I prefer to fetch data as blob and buffer. If it is needed, how does it work when you just have some of the bytes of a multi-byte character in a single chunk? Fails to decode as per the normal algorithm? Any multi-byte characters which are only partially downloaded are not returned in that chunk. I.e. you only expose fully decoded characters. The value in progressEvent.loaded doesn't match up with the accumulated xhr.response.length for multi-byte encodings anyway. This is an interesting case and may be a good reason to have a chunked text type. When I'm parsing streams, I'm going to be waiting on various delimiters, and appending unprocessed bits... So the multibyte issue won't be an issue for me. Consider in any XML format, the developer is going to be seeking closing tags, or such. But, getting the behavior you're looking for, where the callback is ensured that the string has full multibyte characters, that's not particularly easy for developers. While I wouldn't use it, as I'd use chunked arraybuffer, I can see how it'd be useful to achieve those parameters. There are various APIs that expect valid DOMString.
Re: [XHR] chunked
On Wed, Nov 30, 2011 at 12:13 PM, Charles Pritchard ch...@jumis.com wrote: On Nov 30, 2011, at 11:32 AM, Jonas Sicking jo...@sicking.cc wrote: Charles asked whether chunked-text was really needed (and whether we should have chunked which implies ArrayBuffer instead). Nobody got back to him on that. Any text based format would benefit from chunked-text. While the example above uses a binary format, it applies equally to text based formats. And given how much we in this group seem to prefer text based formats, (HTML, CSS, Javascript, EventSource, JSON) I think we should assume that other people at least use them, if not prefer them. My thinking was that ArrayBuffer can easily be converted to String by authors. Even with text-based formats, I prefer to fetch data as blob and buffer. Why? / Jonas
Re: [XHR] chunked
On Wed, Nov 30, 2011 at 1:03 PM, Charles Pritchard ch...@jumis.com wrote: On Nov 30, 2011, at 12:41 PM, Jonas Sicking jo...@sicking.cc wrote: On Wed, Nov 30, 2011 at 12:13 PM, Charles Pritchard ch...@jumis.com wrote: On Nov 30, 2011, at 11:32 AM, Jonas Sicking jo...@sicking.cc wrote: Charles asked whether chunked-text was really needed (and whether we should have chunked which implies ArrayBuffer instead). Nobody got back to him on that. Any text based format would benefit from chunked-text. While the example above uses a binary format, it applies equally to text based formats. And given how much we in this group seem to prefer text based formats, (HTML, CSS, Javascript, EventSource, JSON) I think we should assume that other people at least use them, if not prefer them. My thinking was that ArrayBuffer can easily be converted to String by authors. Even with text-based formats, I prefer to fetch data as blob and buffer. Why? Because it helps with general methods to pass buffers around, and stream processing is buffer based. I don't need to worry about character sets or errant binary data. It's raw and easy to port, being a Transferable object. Why couldn't you use strings instead of buffers when dealing with textual data? And why don't you need need to worry less about characters sets if you are doing the charset conversion rather than the UA? I would imagine you'd have to worry more about character sets if you take on that burden rather than let the UA do it. I can see the argument of being a Transferable object. But why transfer the chunked data between threads? Just do the load on the thread that is going to interpret the data. Many of my encoding and decoding methods expect byte arrays. And these methods are dealing with textual data? Note that no-one is proposing the ability to do chunked-arraybuffer. The question is if we should have chunked-text as well. / Jonas
Re: [XHR] chunked
On 11/30/11 3:44 PM, Jonas Sicking wrote: On Wed, Nov 30, 2011 at 1:03 PM, Charles Pritchardch...@jumis.com wrote: On Nov 30, 2011, at 12:41 PM, Jonas Sickingjo...@sicking.cc wrote: On Wed, Nov 30, 2011 at 12:13 PM, Charles Pritchardch...@jumis.com wrote: On Nov 30, 2011, at 11:32 AM, Jonas Sickingjo...@sicking.cc wrote: Charles asked whether chunked-text was really needed (and whether we should have chunked which implies ArrayBuffer instead). Nobody got back to him on that. Any text based format would benefit from chunked-text. While the example above uses a binary format, it applies equally to text based formats. And given how much we in this group seem to prefer text based formats, (HTML, CSS, Javascript, EventSource, JSON) I think we should assume that other people at least use them, if not prefer them. My thinking was that ArrayBuffer can easily be converted to String by authors. Even with text-based formats, I prefer to fetch data as blob and buffer. Why? Because it helps with general methods to pass buffers around, and stream processing is buffer based. I don't need to worry about character sets or errant binary data. It's raw and easy to port, being a Transferable object. Why couldn't you use strings instead of buffers when dealing with textual data? And why don't you need need to worry less about I can, it just requires extra calls to and from String.fromCharCode and charCodeAt. Using buffers is going to be faster, of course, as they are based on Typed Arrays, whenever I'm doing heavy computation. I do use strings. But for heavy processing and abstracted methods, I use buffers and typed arrays. characters sets if you are doing the charset conversion rather than the UA? I would imagine you'd have to worry more about character sets if you take on that burden rather than let the UA do it. I don't worry about character sets at all. I treat the content as opaque. If there's an error in the character set, that's up to some other part of the process to figure out. I treat the data as fairly opaque. I rarely am trying to sanitize or validate data. I'm just looking for particular values in it. I can see the argument of being a Transferable object. But why transfer the chunked data between threads? Just do the load on the thread that is going to interpret the data. That may often be the case, but when I transfer processed data back to the main thread, I still may want it as a buffer. I've worked a lot on string processing, in old JS and JS with typed arrays. I've certainly ridden the ins and outs and use all methods available. Many of my encoding and decoding methods expect byte arrays. And these methods are dealing with textual data? Note that no-one is proposing the ability to do chunked-arraybuffer. The question is if we should have chunked-text as well. It's a good question, I don't have an answer. I'd sure like an ArrayBuffer to string method. Converting to a Blob then using FileReader, is bit of extra work, as is running fromCharCode over the array to build a string. A conversion method would make chunked-arraybuffer quite easy to convert into a string... Your suggestion that chunked-text would always return complete multi-byte codes was pretty good, as is the notion that whatever chunked-text returns is DOMString-safe, so it'd always work with something like localStorage.setItem. And those are perhaps features that a conversion method would not handle as well,. Sorry if I created confusion or chaos here. Just reporting on what I tend to do. -Charles
Re: [XHR] chunked
On Wed, Nov 30, 2011 at 8:45 PM, Charles Pritchard ch...@jumis.com wrote: I don't worry about character sets at all. I treat the content as opaque. It doesn't sound like you're describing text processing at all, but binary processing of data that happens to contain text. -- Glenn Maynard
Re: [XHR] chunked
On 11/30/11 6:04 PM, Glenn Maynard wrote: On Wed, Nov 30, 2011 at 8:45 PM, Charles Pritchard ch...@jumis.com mailto:ch...@jumis.com wrote: I don't worry about character sets at all. I treat the content as opaque. It doesn't sound like you're describing text processing at all, but binary processing of data that happens to contain text. Right, for chunked encoding, I start by treating it as binary data, and go from there. So, I'm not a good source for the usefulness of chunked-text. When I want text, I convert, but my default is more often to work with buffers. I think Jonas made a good point about multibyte boundaries in text. That's not something that can be solved by a simple method: str = UintToString(IntArrayBufferChunk); But it can be solved simply by delegating it to XHR and a chunked-type response. My only requirement is for chunked-arraybuffer, but I didn't mean to stand in the way of a chunked-text type. -Charles
Re: [XHR] chunked
On Wed, Nov 30, 2011 at 6:16 PM, Charles Pritchard ch...@jumis.com wrote: On 11/30/11 6:04 PM, Glenn Maynard wrote: On Wed, Nov 30, 2011 at 8:45 PM, Charles Pritchard ch...@jumis.com wrote: I don't worry about character sets at all. I treat the content as opaque. It doesn't sound like you're describing text processing at all, but binary processing of data that happens to contain text. Right, for chunked encoding, I start by treating it as binary data, and go from there. So, I'm not a good source for the usefulness of chunked-text. When I want text, I convert, but my default is more often to work with buffers. I think Jonas made a good point about multibyte boundaries in text. That's not something that can be solved by a simple method: str = UintToString(IntArrayBufferChunk); But it can be solved simply by delegating it to XHR and a chunked-type response. My only requirement is for chunked-arraybuffer, but I didn't mean to stand in the way of a chunked-text type. The part of the original email that you responded to was specifically talking about weather we should have chunked-text *as well as* chunked-arraybuffer. So it seems like this sub-thread was off-topic to that question. / Jonas
