Re: Partial object caching

2012-09-19 Thread Bogdan Graur
No more feedback from you guys since one week. Which solution do you consider better? The chunk validity bitmap, the start/end span scheme or maybe another idea? On Wed, Sep 12, 2012 at 1:30 AM, Bogdan Graur wrote: > > > Can we also consider the other way: make the request to the origin server

Re: Partial object caching

2012-09-11 Thread Bogdan Graur
> > Can we also consider the other way: make the request to the origin server > > with a larger range so that we may 'join' two disjoint parts of the data? > > (try to avoid having many empty chunks in between filled chunks) > > We can consider it but I think it won't work. It would require, among

Re: Partial object caching

2012-09-10 Thread Alan M. Carroll
Monday, September 10, 2012, 5:14:40 PM, you wrote: > Can we also consider the other way: make the request to the origin server > with a larger range so that we may 'join' two disjoint parts of the data? > (try to avoid having many empty chunks in between filled chunks) We can consider it but I th

Re: Partial object caching

2012-09-10 Thread Bogdan Graur
> Your write up seems basically accurate to me. I would note again that > reading the earliest Doc of an alternate is an historical artifact of the > implementation and not at all a functional requirement. > I see. I'm glad you cleared this out too. Initially I thought you planned to store the fra

Re: Partial object caching

2012-09-10 Thread Alan M. Carroll
Your write up seems basically accurate to me. I would note again that reading the earliest Doc of an alternate is an historical artifact of the implementation and not at all a functional requirement. The other half of the proposal is that, when servicing a range request that has been forwarded

Re: Partial object caching

2012-09-09 Thread Bogdan Graur
Thank you very much for the clarifications! I think now I have a better view on the current cache structure and your proposal. Current flow for a range request: 1. Find the first Doc for the cached object. 2. The first Doc contains the alternate vector and the fragment table (of course the fragme

Re: Partial object caching

2012-09-05 Thread Alan M. Carroll
Wednesday, September 5, 2012, 4:16:42 PM, you wrote: > Are all Doc structures of the same size? Mostly, but you don't need their actual size, only the size of the content per Doc instance, which is implicit in the fragment offset table. Given a range request we can walk the fragment offset tabl

Re: Partial object caching

2012-09-05 Thread Bogdan Graur
> > > No. The keys are computationally linked. Given a key for a fragment, the > key for the next fragment is computed, not read. OK, I think I found the way the next_key is computed from a given key. > Therefore you can compute the key for fragment i from the key for fragment > 0 without I/O.

Re: Partial object caching

2012-09-05 Thread Alan M. Carroll
Wednesday, September 5, 2012, 3:54:03 AM, you wrote: > Just for clarification: I did some tests and it seems the entries in the > "alternate vector" stored in the first "Doc" are different versions of the > same cached object? Yes, see here: http://trafficserver.apache.org/docs/trunk/sdk/http-hoo

Re: Partial object caching

2012-09-05 Thread Bogdan Graur
> > > 1. Is there a one to many relation between a cached object and a Doc > > structure? > > Yes, that's the chained Docs in the lower right. Each Doc represents a > fragment and we have discussed previously how objects are stored as an > ordered set of fragments. > Just for clarification: I did

Re: Partial object caching

2012-08-30 Thread Alan M. Carroll
Thursday, August 30, 2012, 5:29:16 PM, you wrote: > 1. Is there a one to many relation between a cached object and a Doc > structure? Yes, that's the chained Docs in the lower right. Each Doc represents a fragment and we have discussed previously how objects are stored as an ordered set of frag

Re: Partial object caching

2012-08-30 Thread Bogdan Graur
Thank you for taking the time to draw that diagram! Looking at this diagram and the code some things are not clear for me: 1. Is there a one to many relation between a cached object and a Doc structure? In other words: many Doc structure hold the data for a cached object? The diagram says that on

Re: Partial object caching

2012-08-29 Thread Alan M. Carroll
Kinda sorta of. I have made a simple diagram with some of this information: http://people.apache.org/~amc/ats-cache-layout.jpg which you might find useful. The proposed change would be quite similar to the existing cache structure but would have a couple of major differences - 1) Support having

Re: Partial object caching

2012-08-28 Thread Bogdan Graur
This model of representing a cached object indeed allows holding only partial data for the resource. Is this the current way a cached object is currently represented in TS? On Fri, Aug 24, 2012 at 2:19 PM, Alan M. Carroll < a...@network-geographics.com> wrote: > Thursday, August 23, 2012, 9:00

Re: Partial object caching

2012-08-24 Thread Alan M. Carroll
Thursday, August 23, 2012, 9:00:29 PM, you wrote: > If 1 of N fragments is cached, can we currently serve ranges out of that? No.

Re: Partial object caching

2012-08-23 Thread James Peach
On 23/08/2012, at 5:23 PM, Alan M. Carroll wrote: > Since this has been a topic for a while, I will just throw out an idea to see > how fast you guys can shoot it down. > > A cache object is stored as a series of fragments. If we subdivided each > fragment in to "chunks", we could have 64 chun