Hi, I've noticed that from curl 8.6.0 to 8.7.1, the CPU usage when doing rate-limited downloads with libcurl has increased significantly. This is most noticeable when making multiple (rate-limited) downloads in parallel, for large files. (I've seen this in a custom application using libcurl, and then reproduced using the 'curl' tool.)
For example, running 'curl' with the following (where the file a32 contains 32 copies of " -o /dev/null http://127.0.0.1:9999/speed10";, a test resource with size=46450490): #curl-8.6.0: for x in 1 2 3 4 5 ; do /usr/bin/time ../curl-8_6_0 --parallel --parallel-immediate --no-progress-meter --limit-rate 10485760 `cat ./a32` ; done 5.02 real 0.20 user 0.67 sys 4.95 real 0.15 user 0.68 sys 5.02 real 0.25 user 0.63 sys 5.00 real 0.21 user 0.63 sys 4.49 real 0.23 user 0.62 sys # curl-8.7.1: for x in 1 2 3 4 5 ; do /usr/bin/time ../curl-8_7_1 --parallel --parallel-immediate --no-progress-meter --limit-rate 10485760 `cat ./a32` ; done 4.51 real 0.52 user 0.65 sys 5.01 real 0.55 user 0.63 sys 5.02 real 0.56 user 0.59 sys 4.96 real 0.42 user 0.59 sys 5.01 real 0.37 user 0.71 sys Notice the increase in user-space CPU usage (almost doubled). (Of course the actual effect can vary depending on many factors - the number of parallel downloads, the specified rate limit, file sizes, network speed, etc.) >From testing, this change appears to be due to this commit: https://github.com/curl/curl/commit/db5c9f4f9e0779b49624752b135281a0717b277b That change was to make rate limiting more accurate, and perhaps the increase in CPU usage is a necessary (and expected?) cost of doing so. But, in my use case the rate limiting accuracy in 8.6.0 was "good enough", and I would prefer not to incur the increase in CPU usage seen in 8.7.1. But, I'm not sure how to do that. Perhaps it would be useful for a user of libcurl to be able to (somehow) control this tradeoff between rate-limiting accuracy and CPU usage? Regards, David Pfitzner
-- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
