> I don't think libcurl should do that. Once the transfer is complete, I think > it should return/say so.
But that creates a big problem for back-to-back transfers like it was observed in my examples, for which it looked like there was no rate limit applied at all. And it creates a load problem on the servers which don't see the rate limiting and must serve data with higher load. As far as I know, CPU throttling mechanisms add delays after some operation is completed to prevent scheduling of next operations even though it may not be needed for the already completed operation. And here we have a kind of the same case - we need to add some delay after the transfer to complete to prevent performing the next transfer too soon because we don't have an "ideal" rate limiting mechanism and need to compensate for that after the transfer is done. In 8.6.0, we had almost "ideal" rate limiting mechanism where the speed measurements were done more frequently for the price of higher CPU utilization. But this allowed to perform transfers with rate limiting applied more smoothly and with very high precision. So, if we decreased the number of speed measurements to reduce CPU usage, then we would need to compensate the loss of precision by adding some delays at the end. Otherwise, we can have run-time conditions (like in my test cases) where in some multi-transfer use cases the rate limiting is not actually working, and it will be a regression for clients expecting it to work (and it used to work in 8.17, although not with the same precision as in earlier releases like in 8.6.0). > If your app thinks it needs to add that extra wait, it is really easy for you > to add a sleep there though. Unfortunately, it is not easy. My app is a very large multi-layered framework, where libcurl transfers are used in too many components, including modules closed for modifications, to make it feasible to add additional code which adds delays after each transfer if the rate limit is not observed by the libcurl rate limiting option. And I think I am not alone who has applications like mine. If client code needs to do that kind of actions, then it kind of defeats the purpose of the libcurl rate limit option making it unreliable and unpredictable. And because it will be a regression from the earlier releases, not sure that many folks will be happy about it and the perspective to add additional code to work around the new problem. Thanks! Dmitry -----Original Message----- From: Daniel Stenberg <[email protected]> Sent: Saturday, January 3, 2026 2:36 PM To: Dmitry Karpov via curl-library <[email protected]> Cc: Dmitry Karpov <[email protected]> Subject: [EXTERNAL] Re: Rate limit regressions in libcurl 8.18.0 vs 8.17.0 On Wed, 31 Dec 2025, Dmitry Karpov via curl-library wrote: > As I discussed it with Stefan before, the rate limit mechanism should > apply some small delay at the end of a throttled transfer to maintain > the specified rate limit for the transfer. I don't think libcurl should do that. Once the transfer is complete, I think it should return/say so. > And even though it will not save the bandwidth as all the data has > been already transferred, it will provide proper network speed > measurements for the decision making logic (i.e. bitrate selection > mechanism in video streaming apps) and will help to decrease server > load in back-to-back multi-transfer scenarios. If your app thinks it needs to add that extra wait, it is really easy for you to add a sleep there though. -- / daniel.haxx.se || https://rock-solid.curl.dev -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
