Niklas Edmundsson wrote:
Extra tracking sounds unnecessary if you can do it in a way that doesn't need it.
It's not "extra" it just adding some tracking. When an objects gets cached log (sql, db, whatever) that /blah/foo/bar.html is cached as /cache/x/y/something.meta. Then it's very easy to ask the "store" what is /blah/foo/bar.html cached as? There may be multiples because of vary.
* Clients read from the cache as files are being cached.
That's the hard one, IMO
* Only one session caches the same file.
Easy to do if we use deterministic tmp files and not the way we currently do it. Then all you have to do is when creating temp files use O_EXCL.
* Header/Body updates.
Eaiser with seperate files like mod_disk_cache does now.
* No index/files out-of-sync issues. Ever.
Hard to guarantee, but not impossible. Always to index when storing file and remove when deleting. This should use something like providers so it's not in core cache code and can be easily modified.
With locks, yes it's possible but also a hassle to get right with performance intact.
Not really that hard. Trust me it has been done...
We, as a ftp mirror operated by a non-profit computer club, have a slightly different usecase with single files larger than machine RAM and a working set of approx 40 times larger than RAM. Some bad design decisions in mod_disk_cache becomes really visible in this environment.
Seems to me you should approach problem differently, like rsyncing the mirrored content. I don't know your environment, but was just what I cam up with off the top of my head.
-- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
