RE: [XHR]
Hi Karl, Thanks for weighing in. The issue I was intending to raise was not really parsing XML or JSON or anything like that. It was using chunked delivery of an HTTP response as it is intended to be used -- to allow a client to consume the chunks as they arrive, rather than waiting for the entire response to arrive before using any of it. The requirement to support chunked delivery is specified in section 3.3.1 of RFC 7230. The details of the chunk headers, etc., are contained in section 4.1. Regards, Gomer -- Gomer Thomas Consulting, LLC 9810 132nd St NE Arlington, WA 98223 Cell: 425-309-9933 -Original Message- From: Karl Dubost [mailto:k...@la-grange.net] Sent: Wednesday, March 16, 2016 7:20 PM To: Hallvord R. M. Steen Cc: Gomer Thomas ; WebApps WG Subject: Re: [XHR] Hallvord et al. Le 16 mars 2016 à 20:04, Hallvord Reiar Michaelsen Steen a écrit : > How would you parse for example an incomplete JSON source to expose an > object? Or incomplete XML markup to create a document? Exposing > partial responses for text makes sense - for other types of data > perhaps not so much. I don't think you are talking about the same "parse". The RFC 7230 corresponding section is: http://tools.ietf.org/html/rfc7230#section-4.1 This is the HTTP specification. The content of the specification is about parsing **HTTP** information, not about parsing the content of a body. A JSON, XML, HTML parser is not the domain of HTTP. It's a separate piece of code. Note also for JSON or XML, an incomplete transfert or chunked as text or binary means you can still receive the stream of bytes and choose to serialize it as text or binary, which a JSON or XML processing tool decide to do whatever they want with it. The same way a validating parser would start parsing **something** (as long as it's not completed) and bails out when it finds it invalid. -- Karl Dubost 🐄 http://www.la-grange.net/karl/
RE: [XHR]
Thanks for the information. The "moz-blob" data type looks like it would work, but I need a cross-browser solution in the near future, for new browsers at least. It looks like I might need to fall back on a WebSocket solution with a proprietary protocol between the WebSocket server and applications. The annoying thing is that the W3C XMLHttpRequest() specification of August 2009 contained exactly what I need: The responseBody attribute, on getting, must return the result of running the following steps: If the state is not LOADING or DONE raise an INVALID_STATE_ERR exception and terminate these steps. Return a ByteArray object representing the response entity body or return null if the response entity body is null. Thus, for byteArray data one could access the partially delivered response. For some reason a restriction was added later that removed this capability, by changing "If the state is not LOADING or DONE" to "If the state is not DONE" for all data types except "text". Alas. I still don't understand why W3C and WHATWG added this restriction. Normally new releases of a standard add capabilities, rather than taking them away. It is especially puzzling in this situation, since it basically blows off the IETF RFC 7230 requirement that HTTP clients must support chunked responses. Regards, Gomer -- Gomer Thomas Consulting, LLC 9810 132nd St NE Arlington, WA 98223 Cell: 425-309-9933 -Original Message- From: Jonas Sicking [mailto:jo...@sicking.cc] Sent: Wednesday, March 16, 2016 1:01 PM To: Gomer Thomas Cc: Hallvord Reiar Michaelsen Steen ; WebApps WG Subject: Re: [XHR] Sounds like you want access to partial binary data. There's some propitiatory features in Firefox which lets you do this (added ages ago). See [1]. However for a cross-platform solution we're still waiting for streams to be available. Hopefully that should be soon, but of course cross-browser support across all major browsers will take a while. Even longer if you want to be compatible with old browsers still in common use. [1] https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/responseType / Jonas On Wed, Mar 16, 2016 at 12:27 PM, Gomer Thomas wrote: >In my case the object being transmitted is an ISO BMFF file (as a blob), and I want to be able to present the samples in the file as they arrive, rather than wait until the entire file has been received. > Regards, Gomer > >-- >Gomer Thomas Consulting, LLC >9810 132nd St NE >Arlington, WA 98223 >Cell: 425-309-9933 > > >-Original Message- > From: Hallvord Reiar Michaelsen Steen [mailto:hst...@mozilla.com] > Sent: Wednesday, March 16, 2016 4:04 AM > To: Gomer Thomas > Cc: WebApps WG > Subject: Re: [XHR] > >On Tue, Mar 15, 2016 at 11:19 PM, Gomer Thomas wrote: > >> According to IETF RFC 7230 all HTTP recipients “MUST be able to parse >> the chunked transfer coding”. The logical interpretation of this is >> that whenever possible HTTP recipients should deliver the chunks to >> the application as they are received, rather than waiting for the >> entire response to be received before delivering anything. >> >> In the latest version this can only be done for “text” responses. For >> any other type of response, the “response” attribute returns “null” >> until the transmission is completed. > >How would you parse for example an incomplete JSON source to expose an object? Or incomplete XML markup to create a document? Exposing partial responses for text makes sense - for other types of data perhaps not so much. >-Hallvord > >
RE: [XHR]
In my case the object being transmitted is an ISO BMFF file (as a blob), and I want to be able to present the samples in the file as they arrive, rather than wait until the entire file has been received. Regards, Gomer -- Gomer Thomas Consulting, LLC 9810 132nd St NE Arlington, WA 98223 Cell: 425-309-9933 -Original Message- From: Hallvord Reiar Michaelsen Steen [mailto:hst...@mozilla.com] Sent: Wednesday, March 16, 2016 4:04 AM To: Gomer Thomas Cc: WebApps WG Subject: Re: [XHR] On Tue, Mar 15, 2016 at 11:19 PM, Gomer Thomas wrote: > According to IETF RFC 7230 all HTTP recipients “MUST be able to parse > the chunked transfer coding”. The logical interpretation of this is > that whenever possible HTTP recipients should deliver the chunks to > the application as they are received, rather than waiting for the > entire response to be received before delivering anything. > > In the latest version this can only be done for “text” responses. For > any other type of response, the “response” attribute returns “null” > until the transmission is completed. How would you parse for example an incomplete JSON source to expose an object? Or incomplete XML markup to create a document? Exposing partial responses for text makes sense - for other types of data perhaps not so much. -Hallvord
RE: [XHR]
Thanks for the suggestion.
I looked at the Streams specification, and it seems pretty immature and
underspecified. I’m not sure it is usable by someone who doesn’t already know
how it is supposed to work before reading the specification. How many of the
major web browsers are supporting it?
For example:
(1)The constructor of the ReadableStream object is “defined” by
Constructor (underlyingSource = { }, {size, highWaterMark = 1 } = { } )
The “specification” states that the underlyingSource object “can” implement
various methods, but it does not say anything about how to create or identify a
particular underlyingSource. In my case I want to receive a stream from a
remote HTTP server. What do I put in for the underlyingSource? What does the
underlyingSource on the remote server need to do? Also, what does the
“highWaterMark” parameter mean? The “specification” says it is part of the
queuing strategy object, but it does not say what it does. Is it the maximum
number of bytes of unread data in the Stream? If so, it should say so. If the
“size” parameter is omitted, is the underlyingSource free to send chunks of any
size, including variable sizes?
(2)The ReadableStream class has a “getReader()” method, but the
specification gives no hint as to the data type that this method returns. I
suspect that it is an object of the ReadableStreamReader class, but if so it
would be nice if the “specification” said so.
Regards, Gomer
--
Gomer Thomas Consulting, LLC
9810 132nd St NE
Arlington, WA 98223
Cell: 425-309-9933
From: Jonathan Garbee [mailto:jonathan.gar...@gmail.com]
Sent: Wednesday, March 16, 2016 5:10 AM
To: Hallvord Reiar Michaelsen Steen ; Gomer Thomas
Cc: WebApps WG
Subject: Re: [XHR]
If I understand correctly, streams [1] with fetch should solve this use-case.
[1] https://streams.spec.whatwg.org/
On Wed, Mar 16, 2016 at 7:10 AM Hallvord Reiar Michaelsen Steen
mailto:hst...@mozilla.com> > wrote:
On Tue, Mar 15, 2016 at 11:19 PM, Gomer Thomas
mailto:go...@gomert-consulting.com> > wrote:
> According to IETF RFC 7230 all HTTP recipients “MUST be able to parse the
> chunked transfer coding”. The logical interpretation of this is that
> whenever possible HTTP recipients should deliver the chunks to the
> application as they are received, rather than waiting for the entire
> response to be received before delivering anything.
>
> In the latest version this can only be done for “text” responses. For any
> other type of response, the “response” attribute returns “null” until the
> transmission is completed.
How would you parse for example an incomplete JSON source to expose an
object? Or incomplete XML markup to create a document? Exposing
partial responses for text makes sense - for other types of data
perhaps not so much.
-Hallvord
RE: [XHR]
Hi Domenic,
Thanks for your response. Please see my embedded remarks below
(labeled with [GT]).
Regards, Gomer
--
Gomer Thomas Consulting, LLC
9810 132nd St NE
Arlington, WA 98223
Cell: 425-309-9933
-Original Message-
From: Domenic Denicola [mailto:d...@domenic.me]
Sent: Thursday, March 17, 2016 11:56 AM
To: Gomer Thomas ; 'Jonathan Garbee'
; 'Hallvord Reiar Michaelsen Steen'
Cc: 'WebApps WG'
Subject: RE: [XHR]
From: Gomer Thomas [mailto:go...@gomert-consulting.com]
> I looked at the Streams specification, and it seems pretty
immature and underspecified. I’m not sure it is usable by someone who doesn’t
already know how it is supposed to work before reading the specification. How
many of the major web browsers are supporting it?
Thanks for the feedback. Streams is intended to be a lower-level
primitive used by other specifications, primarily. By reading it you're
supposed to learn how to implement your own streams from basic underlying
source APIs.
[GT] It would be good to say this in the specification, and
reference some sample source APIs. (This is an example of what I meant when I
said it is very difficult to read the specification unless one already knows
how it is supposed to work.)
> (1) The constructor of the ReadableStream object is “defined”
by
> Constructor (underlyingSource = { }, {size, highWaterMark = 1 }
= { }
> ) The “specification” states that the underlyingSource object
“can”
> implement various methods, but it does not say anything about
how to
> create or identify a particular underlyingSource
As you noticed, specific underlying sources are left to other
places. Those could be other specs, like Fetch:
https://fetch.spec.whatwg.org/#concept-construct-readablestream
or it could be used by authors directly:
https://streams.spec.whatwg.org/#example-rs-push-no-backpressure
> In my case I want to receive a stream from a remote HTTP
server. What do I put in for the underlyingSource?
This is similar to asking the question "I want to create a
promise for an animation. What do I put in the `new Promise(...)` constructor?"
In other words, a ReadableStream is a data type that can stream anything, and
the actual capability needs to be supplied by your code. Fetch supplies one
underlying source, for HTTP responses.
> Also, what does the “highWaterMark” parameter mean? The
“specification” says it is part of the queuing strategy object, but it does not
say what it does.
Hmm, I think the links (if you follow them) are fairly clear.
https://streams.spec.whatwg.org/#queuing-strategy. Do you have any suggestions
on how to make it clearer?
[GT] I did follow the link before I sent in my questions. In
section 2.5 it says "The queuing strategy assigns a size to each chunk, and
compares the total size of all chunks in the queue to a specified number, known
as the high water mark. The resulting difference, high water mark minus total
size, is used to determine the desired size to fill the stream’s queue." It
appears that this is incorrect. It does not seem to jibe with the default value
and the examples. As far as I can tell from the default value and the examples,
the high water mark is not the total size of all chunks in the queue. It is the
number of chunks in the queue. Also, this is somewhat problematic as a measure
unless the chunks are uniform in size. If the chunks are required to all be the
same size, this greatly reduces the usefulness of the Streams concept.
> Is it the maximum number of bytes of unread data in the Stream?
If so, it should say so.
Close; it is the maximum number of bytes before a backpressure
signal is sent. But, that is already exactly what the above link (which was
found by clicking the links "queuing strategy" in the constructor definition)
says, so I am not sure what you are asking for.
> If the “size” parameter is omitted, is the underlyingSource
free to send chunks of any size, including variable sizes?
Upon re-reading, I agree it's not 100% clear that the size()
function maps to "The queuing strategy assigns a size to each chunk". However,
the behavior of how the stream uses the si
[XHR]
Dear Colleagues, The XHR specification has one very unsatisfactory aspect. It appears that W3C and WHATWG are snubbing your noses at IETF. According to IETF RFC 7230 all HTTP recipients "MUST be able to parse the chunked transfer coding". The logical interpretation of this is that whenever possible HTTP recipients should deliver the chunks to the application as they are received, rather than waiting for the entire response to be received before delivering anything. In earlier versions of the XMLHttpRequest() specification, this was possible. The various forms of the "response" attribute (for different response types) could be retrieved at any time during the transmission, to get the portion of the response that had been received up to that point. In the latest version this can only be done for "text" responses. For any other type of response, the "response" attribute returns "null" until the transmission is completed. This is a very unfortunate change. There are applications for which it is extremely valuable to be able to acquire partial results as the transmission progresses. I hope you will make the minimal changes in the specification that will allow partial results to be accessed during transmission for all response types, not just text responses. Regards, Gomer Thomas -- Gomer Thomas Consulting, LLC 9810 132nd St NE Arlington, WA 98223 Cell: 425-309-9933
