On Fri, 2003-10-31 at 10:57, Henrik Nordstrom wrote: > On Fri, 31 Oct 2003, Robert Collins wrote: > > > > * store merge logics when there is multiple data feeds (different clients > > > requesting different not yet cached ranges) > > > > Heh, I actually consider this a reason to feed into one store object > > rather than a new one. > > before we continue this we really need to have the discussion clearly > separated in > > * client-side > * server-side > * store > * broker > > it is quite evident that both of us are very unclear on the split between > broker and store.
yah. > > Sure. The issue I have with your design is that it makes the > > server/store drive everything which is exactly the problem we had before > > where the client side drove everything. Downgrading is esaier if the > > client side can drive it's needs. Teaching the store about freeable > > ranges in random access needs to be done - in either approach. > > Why I propose what I do is because there is a high risk that we will be > receiving reply content in a different order than asked for. When > designing for processing retrieved ranged HTTP data it must be assumed > that from time to time the returned data will be compatible with what was > asked for but not in the exact same order. The most probable cases (save > for for non-ranged response) is that the ranges have been simplified or > sorted in the response. > > This brings the issue of receiving a fully valid reply, but which does not > fulfill our requirements in order to be able to construct a suitable reply > to the client if it is the client who decides on what it wants to see. > This is not so much of a problem when not doing range merging, but when > doing range merging it is a more difficult problem as some times it may > not be evident from start that there is a problem. In which case we need to hold off telling the client anything until we have enough info to see there won't be a consistency problem. > If you look closely at what I propose the result is in short that on cache > misses (full or partial) the client side has no control over the order of > the returned data. It is up to the server side (ours, or origin server) to > make sure the content arrives in an order compatible with the request. mmm, probably, haven't looked at all the corner cases. > only on cache hits can the client side have control, but in order to unify > the two cases this control is isolated to just telling at start what is > wanted. > My ideas in more detail: > > > Client -> Broker > > I want these ranges. > I accept/don't accept multi-part responses. (can be deduced from the > wanted ranges) Right, easily handed in via the request->range pointer. > Broker -> store > > Random read/write, as cache-able data flows via the broker. > > non-cache-able data never hits the store at all. > > Broker -> Server side (miss, full or partial) > > Client wants these ranges > Clients accepts / don't accept multi-part responses. (can be deduced from > the wanted ranges) > I have these entities (and ranges thereof) in the store which may be > compatible with the request. This information is used for constructing > If-Range, If-None-Match etc. How do you propose to handle entities the client has that we don't? > Related Note: In some cases two server round-trips may be required by > the server side to satisfy the request, first If-None-Match to find > the correct reply entity, then followed by If-Range if the entity is not > complete. > > Server side -> Broker > > Reply entity headers > Ranges of entity body > > Broker -> Client > > Entity headers > Ranges of entity body. In case of cache misses the ranges are returned > in the exact same order as received from upstream (server side). which may break on merging for single part replies. > About the only thing I have not yet set my mind on is the tiny detail > where to do range merging. This is either done server-side or in the > broker. My initial preference is for the server-side as it makes it easier > to deal with corner cases, but more efficient in the broker as the parts > who are cache hits don't need to be looped back and also allows for store > merging in place without creating a new store object. I think the broker is the place. > > Yeah. Theres other uses for the notification - to implement the arrival > > of headers, and for the transactions needed for 1XX Continue support. > > Headers should be given by a completely different "read" operation in my > opinion. The whole forwarding and store paths should be strict about > separation between headers and body type content. This is true both for > initial headers and trailer header data, and the presence of trailer > header data further emphasizes the reasons why the store needs to separate > the two cleanly. Agreed. Rob -- GPG key available at: <http://members.aardvark.net.au/lifeless/keys.txt>.
signature.asc
Description: This is a digitally signed message part
