On Wed, Feb 22, 2023 at 07:23:42PM +0100, Stefan Eissing via curl-library wrote: > There is no one size fits all here, as Daniel indicates. If you > register callbacks in curl to get your answers and expect them to be > called on a particular thread, you need to run curl in that thread. > > The transfer sizes matter as well. If you have many small transfers, > multiplexing can really help. For large transfers, the gains are > less. But then again, if you, have a small request in the "middle" > it will not need to wait on the large ones to complete. > > As you can see, it really depends on your application, how it > processes the responses, what kind of requests it makes and what > response times it needs.
Thanks, and yes this is a very interesting point. I only implied it in my original question, but we are using small HTTP range requests (CURLOPT_RANGE). The request size depends, but typically you'd see requests ranging from the qcow2 cluster size (32K) up to the nbdcopy default request size (256K). In theory it could be [512b..32M]. The idea behind this is that we can boot a machine off a web server, downloading only the sectors/clusters as needed (with a lot of caching going on). In more advanced cases we're able to inspect a remote disk image, determine which parts are unused (by using filesystem metadata within the image), and then download only the parts which contain data, even modifying the disk image filesystem content in flight, and do all of this while streaming and making no local copies. It's quite an interesting research area. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
