On Sat, Jan 05, 2008 at 05:25:06PM +0300, Dmitry Kurochkin wrote: > 2008/1/5, Tristan Seligmann <[EMAIL PROTECTED]>: > > * Dmitry Kurochkin <[EMAIL PROTECTED]> [2008-01-05 02:58:35 +0300]: > > > > > > Also related to this idea: can we adjust the order of downloads in the > > > > queue? e.g. maybe I'd like to add a file towards the front of the queue > > > > because I need it right now. This might be doable if waitForURL could > > > > bump > > > > up the priority of that URL, in case it hasn't yet been requested from > > > > the > > > > server. > > > Implementing priority is more difficult. There is no priority support > > > in curl AFAIK. Maybe there is priority in libwww but I do not think > > > > Wouldn't the use of pipelining mean that all of the requests should have > > already been sent? > > No. Requests are sent by some constant amount (say 5) at a time.
I suppose perhaps we could handle priority ourselves, and then feed curl the requests in priority order? Of course, we don't *need* priority queuing, but it affects how simplistically we can write the code that sends out requests for files before we need them. If we lack anything like priority queueing, we'd have to cautiously only speculatively request a few files at a time. This does have the advantage of also avoiding too much uneeded downloading, but it also means that we'll have to add files to the queue more often. > > > Regarding the priority. Maybe we can get what something like this with > > > a second connection to the server? One connection for high priority > > > > This doesn't seem like a completely awful idea, but there should > > probably be a way to disable this, as some HTTP servers do not want to > > accept more than one connection at a time from the same source (among > > other potential problems). > > We can fall back to a single connection if second connection fails. > Also we have an explicit option for this. I don't think I like this approach, as it doesn't really emulate the priority queue. In particular, we would probably have no way to avoid duplication of effort between the two connections... unless you could pull an easy handle out of a multi handle and move it to a different multi handle... which sounds fishy. -- David Roundy Department of Physics Oregon State University _______________________________________________ darcs-devel mailing list [email protected] http://lists.osuosl.org/mailman/listinfo/darcs-devel
