Andrew is correct. It is not necessary for every chunk to be the same length. Each chunk must begin with the size of the chunk. Chunked, or interactive responses are a feature of HTTP 1.1, which not all browsers fully implement. IE6, for example, basically does not handle this behavior at all. Modern browsers do though, so it might be a useful thing to implement.
This is actually quite easy to do in JavaScript. In the onreaderstatechange function simply check for readystate 3 instead of 4. The 3 state is the "interactive" state and is set when the browser receives one of these chunks without the server closing the connection. Unfortunately, getting a server to correctly handle this behavior is much harder. Most do not allow you to explicitely send chunks. As stated above the flush() function does not necessarily send the data at that moment. I actually had to write a web server from scratch to get the behavior that I wanted. Server side development isn't the focus of this list though, so it seems reasonable to add the ability for jQuery to handle a readystate 3 properly. It would basically need to act as it does currently, but without clearing the statechange function as it will be called again. There is no way to handle this behavior, afaik, with script injection. Sean On May 31, 12:09 pm, Andrew Ingram <[email protected]> wrote: > From what I can gather, it's not a requirement of the protocol that > each chunk be the same length (though it is a common implementation). > In Django, for example, to do a chunked response you simply return an > iterator to the HttpResponse constructor. I've built examples where > each element in the iterator was a difference size and it still seemed > to work. > > If you deliver chunks so that each one has a common format, rather > than a fixed length, it should be completely possible to handle each > chunk as it arrives. The limiting factor might be that XMLHttpRequest > objects won't return any response until it has arrived in its > entirety, but I'm more knowledgable about generating these responses > than handling them which is why I'm raising the idea here. > > Example: > > Chunk 1 > <div class="response-item"> > ... as much as you want here ... > </div > > Chunk 2 > <div class="response-item"> > ... potentially a different amount of data to chunk 1... > </div> > > You'd than have to tell the AJAX method how to recognise each > individual chunk, possible using selector syntax in the case of an XML > response or maybe object graph notation for JSON. Then as each chunk > is received it would be fired off to your callback function. > > Again, I anticipate that the road-block may lie in AJAX being unable > to handle partial responses, but given Digg's experimentation with > this (though I'm not sure how they do it), I'm hopeful that it may > actually be possible. > > - Andrew > > On May 31, 4:49 pm, Diego Perini <[email protected]> wrote: > > > > > Andrew, > > this is a neat idea but from my knowledge it is not currently > > applicable to HTML fragments to be inserted in the DOM. > > > "Transfer-encoding: chunked" means that "output_buffering" is enabled > > on the server. Each chunk size is predefined (configurable), normally > > the default in PHP is set to 4Kbytes but could be made smaller or > > larger, the configuration in PHP may be changed by means of the > > "output_buffering" directive. > > > Here are the relevant commented lines from my "php.ini": > > > ; Output buffering allows you to send header lines (including cookies) > > even > > ; after you send body content, at the price of slowing PHP's output > > layer a > > ; bit. You can enable output buffering during runtime by calling the > > output > > ; buffering functions. You can also enable output buffering for all > > files by > > ; setting this directive to On. If you wish to limit the size of the > > buffer > > ; to a certain size - you can use a maximum number of bytes instead of > > 'On', as > > ; a value for this directive (e.g., output_buffering=4096). > > output_buffering = On > > > The problem is that for HTML we have to insert the fragment in the DOM > > and to do that we need to know where the HTML tags starts and where > > they end, we could not insert a "chunk" like this one: > > > <div>Test example <strong>Reults (no closing tag has arrived in the > > last chunk, so broken HTML) > > > Yes that's what can happen, since the chunks have a predefined length > > we have no way to know the size in advance from the client-side, maybe > > something could be done server-side to mitigate this problem with HTML > > data (some wrapper/parser). > > > If the data has no structure, and can be rendered or used byte by byte > > then maybe this idea has a bright future. > > > I confirm that "Transfer-encoding: chunked" is not MXHR (Multi-part > > XHR) like in DUI.Stream, it is just a mean of transferring long > > streams of data in chunks as the answer to a single HTTP request. And > > yes if feasible it will perform better than MXHR since the multi-part > > headers and related TCP overhead will be avoided both sides and > > Cookies will be sent only once in the first and only XHR. > > > +1 for the idea, keep us informed on the development. > > > Diego > > > PS: "Transfer-encoding: chunked" and browser local caching is what > > makes our life difficult in detecting the DOMContentLoaded event in > > IE. In fact in this mode the client has no way of knowing in advance > > the length of the answer, if transfer is chunked there will not be a > > "Content-Length: xxxx" header in the answer. > > > On 30 Mag, 22:26, Andrew Ingram <[email protected]> wrote: > > > > I'm talking about content-encoding: chunked, it's a single connection. > > > The server can periodically 'flush' the response stream to send all > > > content that has been generated up to that point. The idea being that > > > if each chunk is self-contained, ie a single entity on a response that > > > would return a list of entities, the AJAX library can handle the first > > > entities before the complete response has even finished being > > > generated by the server. > > > > Technically this could require even less connection overhead than > > > Comet because you could keep the connection open after delivering each > > > 'update' - rather than requiring the client to create a new connection > > > each time. > > > >http://en.wikipedia.org/wiki/Chunked_transfer_encoding > > > > Now I could be completely mistaken and actually chunked encoding > > > requires multiple connections, but I don't believe this to be the > > > case. > > > > On May 30, 3:31 pm, Ricardo <[email protected]> wrote: > > > > > Creating lots of connections would probably have a large overhead > > > > making it slower than if you waited for the whole processing to end, > > > > for each connection you have to factor the 2-way latency + server > > > > response time. A better approach and already usable is HTTP Streaming/ > > > > Comet: > > > > >http://ajaxpatterns.org/HTTP_Streaming > > > > > On May 29, 7:36 pm, Andrew Ingram <[email protected]> wrote: > > > > > > Hi all, > > > > > > I'm not even sure if this is possible with JavaScript at the moment, > > > > > but it would make a powerful feature if it were. > > > > > > If returning a list of resources as the response to a request, it's > > > > > relatively trivial to configure the app (in Django at least) to flush > > > > > the stream after each resource and provide a semi real-time feed of > > > > > results, ie you don't have to wait for the last item to be calculated > > > > > before the first one is returned. This uses Content-Encoding chunked. > > > > > > I was thinking that if jQuery could somehow recognise these types of > > > > > response, it could iterate over these individual resources as they > > > > > come over the wire, then the callback would be given individual items > > > > > rather than the full response. This would make AJAXy functionality > > > > > even more responsive because you can start handling parts of the > > > > > response before the server has even finished generating the later > > > > > parts. > > > > > > Maybe this is already possible, but I couldn't have any documentation > > > > > or mention of it. > > > > > > Any thoughts on this idea? > > > > > > Regards, > > > > > Andrew Ingram --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "jQuery Development" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/jquery-dev?hl=en -~----------~----~----~----~------~----~------~--~---
