On Fri, 31 Oct 2003, Robert Collins wrote: > > * store merge logics when there is multiple data feeds (different clients > > requesting different not yet cached ranges) > > Heh, I actually consider this a reason to feed into one store object > rather than a new one.
before we continue this we really need to have the discussion clearly separated in * client-side * server-side * store * broker it is quite evident that both of us are very unclear on the split between broker and store. > > If/when we start doing merging of partial responses then there will be > > situations where such invalid responses is not compatible with the > > request. > > True - and thats where http + store logics will belong, to fallback to > unconditional upstream retrieval when needed. Agreed. > Sure. The issue I have with your design is that it makes the > server/store drive everything which is exactly the problem we had before > where the client side drove everything. Downgrading is esaier if the > client side can drive it's needs. Teaching the store about freeable > ranges in random access needs to be done - in either approach. Why I propose what I do is because there is a high risk that we will be receiving reply content in a different order than asked for. When designing for processing retrieved ranged HTTP data it must be assumed that from time to time the returned data will be compatible with what was asked for but not in the exact same order. The most probable cases (save for for non-ranged response) is that the ranges have been simplified or sorted in the response. This brings the issue of receiving a fully valid reply, but which does not fulfill our requirements in order to be able to construct a suitable reply to the client if it is the client who decides on what it wants to see. This is not so much of a problem when not doing range merging, but when doing range merging it is a more difficult problem as some times it may not be evident from start that there is a problem. If you look closely at what I propose the result is in short that on cache misses (full or partial) the client side has no control over the order of the returned data. It is up to the server side (ours, or origin server) to make sure the content arrives in an order compatible with the request. only on cache hits can the client side have control, but in order to unify the two cases this control is isolated to just telling at start what is wanted. My ideas in more detail: Client -> Broker I want these ranges. I accept/don't accept multi-part responses. (can be deduced from the wanted ranges) and when asked by server side: request entity data (which also triggers 100 Continue if needed in order to get this from the client) Broker -> store Random read/write, as cache-able data flows via the broker. non-cache-able data never hits the store at all. Broker -> Server side (miss, full or partial) Client wants these ranges Clients accepts / don't accept multi-part responses. (can be deduced from the wanted ranges) I have these entities (and ranges thereof) in the store which may be compatible with the request. This information is used for constructing If-Range, If-None-Match etc. Related Note: In some cases two server round-trips may be required by the server side to satisfy the request, first If-None-Match to find the correct reply entity, then followed by If-Range if the entity is not complete. Server side -> Broker Reply entity headers Ranges of entity body Broker -> Client Entity headers Ranges of entity body. In case of cache misses the ranges are returned in the exact same order as received from upstream (server side). About the only thing I have not yet set my mind on is the tiny detail where to do range merging. This is either done server-side or in the broker. My initial preference is for the server-side as it makes it easier to deal with corner cases, but more efficient in the broker as the parts who are cache hits don't need to be looped back and also allows for store merging in place without creating a new store object. > Yeah. Theres other uses for the notification - to implement the arrival > of headers, and for the transactions needed for 1XX Continue support. Headers should be given by a completely different "read" operation in my opinion. The whole forwarding and store paths should be strict about separation between headers and body type content. This is true both for initial headers and trailer header data, and the presence of trailer header data further emphasizes the reasons why the store needs to separate the two cleanly. Regards Henrik
