I ran the same throttling tests on 8.16.0 that helped to find the speed throttling issue on 8.7, and I can tell that the throttling accuracy in 8.16.0 is significantly worse than in 8.6.0 and 8.7.1.
My tests use 1MB downloads on network speeds: 700mbps, 50mbps and 20mbps, with 8mbps and 16mbps throttling speeds. The test measures the performed transfers speeds on 10 downloads and calculates average and max deviations from the rate limits. Here is the brief description of test results: 8.6.0 (and 8.7.1) Network speed: 700mbps, Rate limit: 8mbps, Avg deviation: -0.86%, Max deviation: -1.5%. Network speed: 700mbps, Rate limit: 16mbps, Avg deviation: -1.72%, Max deviation: -2.8%. Network speed: 50mbps, Rate limit: 8mbps, Avg deviation: -0.37%, Max deviation: -1%. Network speed: 50mbps, Rate limit: 16mbps, Avg deviation: - 1.64%, Max deviation: -3.2%. Network speed: 20mbps, Rate limit: 8mbps, Avg deviation: -0.6%, Max deviation: -1.1%. Network speed: 20mbps, Rate limit: 16mbps, Avg deviation: -1.37%, Max deviation: -1.7%. 8.16.0 Network speed: 700mbps, Rate limit: 8mbps, Avg deviation: 4.65%, Max deviation: 13.7%. Network speed: 700mbps, Rate limit: 16mbps, Avg deviation: 6.69%, Max deviation: 17.7%. Network speed: 50mbps, Rate limit: 8mbps, Avg deviation: 7.03%, Max deviation: 10.2%. Network speed: 50mbps, Rate limit: 16mbps, Avg deviation: 9.99%, Max deviation: 15%. Network speed: 20mbps, Rate limit: 8mbps, Avg deviation: 10.89%, Max deviation: 15.5%. Network speed: 20mbps, Rate limit: 16mbps, Avg deviation: 5.34%, Max deviation: 14.5%. As we can see, the 8.6.0 (and 8.7.1) has almost perfect rate limit accuracy with a tendency to curb the download speed a little but below the limit, which is a good thing that makes the download speed unlikely to go beyond the limit. On the other hand, the 8.16.0 has wider deviations from the rate limits with a tendency to use more network bandwidth than the limit specifies. And the max deviations above 10% make it difficult to have reliable predictions of the throttled download speeds. If the almost perfect accuracy of 8.6.0/8.7.1 comes with a price of much higher CPU utilization, which maybe not acceptable for everyone, then maybe we need to adjust the rate limiting mechanism and make it to provide just 5% accuracy with a tendency to use less network bandwidth than the rate limit, when the precise accuracy is not possible. The tendency to use more bandwidth in the current mechanism creates problems when rate limits are specifically set to prevent low priority downloads from stealing network bandwidth from downloads with higher priority. Thanks, Dmitry Karpov -----Original Message----- From: curl-library <[email protected]> On Behalf Of Dmitry Karpov via curl-library Sent: Tuesday, October 28, 2025 11:38 AM To: libcurl development <[email protected]> Cc: Dmitry Karpov <[email protected]> Subject: [EXTERNAL] Re: CPU usage since 8.7.1 It was me who reported it back then. The precision loss of the rated downloads was more than 10% in some cases, which was above our 5% tolerance. It caused problems in multi-download prioritized scenarios where rated downloads with less priority were stealing bandwidth from the downloads with higher priority. I am also concerned about CPU usage, but 10% rate precision deviation is too high. And in our case (video streaming application), the problem was that the prioritized video got worse quality and other problems which affected our users. Ideally, the optimal solution would be to avoid busy loops and using too many CPU cycles, and provide 5% rate accuracy. Thanks! Dmitry Karpov -----Original Message----- From: curl-library <[email protected]> On Behalf Of Gleb Smirnoff via curl-library Sent: Tuesday, October 28, 2025 11:11 AM To: Daniel Stenberg <[email protected]>; Stefan Eissing <[email protected]> Cc: Gleb Smirnoff <[email protected]>; Gleb Smirnoff via curl-library <[email protected]> Subject: [EXTERNAL] Re: CPU usage since 8.7.1 On Tue, Oct 28, 2025 at 05:32:27PM +0100, Daniel Stenberg wrote: D> On Tue, 28 Oct 2025, Gleb Smirnoff via curl-library wrote: D> D> > So here I am raising this question. What needs to be done to get D> > CPU savings of 8.6.0 level and how can we help here? D> D> As recall things, the change that introduced the new (current) D> behavior was done to enhance the accuracy of the rate limit logic, at D> the expense of CPU. D> D> The change was pushed for by someone who argued that the increased D> precision was valuable for them if I remember correctly. D> D> Can we find a way to have both ways and let the user select which? D> Maybe for example by maybe something as silly and simple as doing D> CURLOPT_MAX_RECV_SPEED_LARGE % 10000 and if that is non-zero, higher D> precision is wanted. I added Stefan to explicit Cc. Since he very recently added 24badd29f titled "limit-rate revisited" (of course this conlicted with out patch again), he definitely should have an opinion on how this can be solved. -- Gleb Smirnoff -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
