On Tue, May 18, 2010 at 08:32:32PM -0500, Shawn Walker wrote: > I would note that Apache's mod_cache and mod_disk_cache appear to > properly handle byte-range requests.
Ok, cool. That's good to know. > > In the case where we're performing parallel file requests, we can > > spread the I/O out amongst servers behind the load-balancer, which > > helps. But any operation that gets a bunch of small files is going to > > be slower than something streaming, whether that's file/0, a byte > > range request, or filelist. > > While I agree with the things you've pointed out to a certain > extent, my limited understanding of other protocols (such as > bittorrent, etc.) indicates they essentially completely (or can) > operate on a set of byte range requests and obviously can provide > substantial aggregate transfer performance. So with that in mind, > even if we can't necessarily use byte-range requests for http, > perhaps we can use it for other protocols such as ftp, torrent, etc. > ? I think I said this before, but perhaps I was simply unclear. I think that having the ability to perform byte-range requests is useful, but I'm not sure that it fits with our current use and deployment cases right now. I do agree that using byte-range HTTP requests could be important if our usage model changes, or we have customers who hit a particular edge case that our current approach doesn't handle well. I'm having a hard time imagining what case that would be now, but I'd hate to preclude future work based upon my own lack of imagination. > I've also toyed with the idea of allowing the client to use an > archive file as an origin in a network scenario. While dynamic > operations wouldn't be possible, there's a certain attractiveness of > putting up a single giant archive file instead of extracting a few > million files somewhere. Yes, that's essentially what I've been talking about for the case where the client wants to download the entire archive, instead of each file individually. If the archive files are big enough, they might also be candidates for bit-torrent, or some other byte-range protocol, like you mentioned in your previous paragraph. -j _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
