Hi Stefan, I tried your updated PR, and it seems that it "overthrottles" significantly on all rate limits. Hopefully, you can fine tune it a little bit more and get a bit less "overthrottling", and I will be happy to test it again.
Thanks, Dmitry Here are the stats for 1MB test: 1 MB downloads ################################################################################### Speed limit test [LAN (~700 mbps)], iterations=5 Url=http://192.168.1.218/Data/file_1M.bin, max_speed=8000000 bps time=1162 ms, dnld=1048576 B, speed=7217515 bps, spd_diff=-782485 bps, pct=-9.8 % time=1121 ms, dnld=1048576 B, speed=7480851 bps, spd_diff=-519149 bps, pct=-6.5 % time=1134 ms, dnld=1048576 B, speed=7396800 bps, spd_diff=-603200 bps, pct=-7.5 % time=1114 ms, dnld=1048576 B, speed=7528708 bps, spd_diff=-471292 bps, pct=-5.9 % time=1120 ms, dnld=1048576 B, speed=7488892 bps, spd_diff=-511108 bps, pct=-6.4 % -------------------------------------------------------------------- avg_deviation=-7.2 %, max_deviation=-9.8 % Url=http://192.168.1.218/Data/file_1M.bin, max_speed=16000000 bps time=606 ms, dnld=1048576 B, speed=13820670 bps, spd_diff=-2179330 bps, pct=-13.6 % time=636 ms, dnld=1048576 B, speed=13186898 bps, spd_diff=-2813102 bps, pct=-17.6 % time=618 ms, dnld=1048576 B, speed=13573140 bps, spd_diff=-2426860 bps, pct=-15.2 % time=592 ms, dnld=1048576 B, speed=14154834 bps, spd_diff=-1845166 bps, pct=-11.5 % time=638 ms, dnld=1048576 B, speed=13129890 bps, spd_diff=-2870110 bps, pct=-17.9 % -------------------------------------------------------------------- avg_deviation=-15.2 %, max_deviation=-17.9 % Url=http://192.168.1.218/Data/file_1M.bin, max_speed=20000000 bps time=496 ms, dnld=1048576 B, speed=16895212 bps, spd_diff=-3104788 bps, pct=-15.5 % time=512 ms, dnld=1048576 B, speed=16383808 bps, spd_diff=-3616192 bps, pct=-18.1 % time=506 ms, dnld=1048576 B, speed=16556386 bps, spd_diff=-3443614 bps, pct=-17.2 % time=510 ms, dnld=1048576 B, speed=16421429 bps, spd_diff=-3578571 bps, pct=-17.9 % time=494 ms, dnld=1048576 B, speed=16952436 bps, spd_diff=-3047564 bps, pct=-15.2 % -------------------------------------------------------------------- avg_deviation=-16.8 %, max_deviation=-18.1 % Url=http://192.168.1.218/Data/file_1M.bin, max_speed=32000000 bps time=366 ms, dnld=1048576 B, speed=22919130 bps, spd_diff=-9080870 bps, pct=-28.4 % time=303 ms, dnld=1048576 B, speed=27633100 bps, spd_diff=-4366900 bps, pct=-13.6 % time=304 ms, dnld=1048576 B, speed=27572518 bps, spd_diff=-4427482 bps, pct=-13.8 % time=318 ms, dnld=1048576 B, speed=26365921 bps, spd_diff=-5634079 bps, pct=-17.6 % time=301 ms, dnld=1048576 B, speed=27827342 bps, spd_diff=-4172658 bps, pct=-13.0 % -------------------------------------------------------------------- avg_deviation=-17.3 %, max_deviation=-28.4 % ################################################################################### Speed limit test [50 mbps], iterations=5 Url=http://192.168.1.218/Data/file_1M.bin, max_speed=8000000 bps time=1147 ms, dnld=1048576 B, speed=7310760 bps, spd_diff=-689240 bps, pct=-8.6 % time=1132 ms, dnld=1048576 B, speed=7405884 bps, spd_diff=-594116 bps, pct=-7.4 % time=1124 ms, dnld=1048576 B, speed=7461461 bps, spd_diff=-538539 bps, pct=-6.7 % time=1118 ms, dnld=1048576 B, speed=7497768 bps, spd_diff=-502232 bps, pct=-6.3 % time=1141 ms, dnld=1048576 B, speed=7349718 bps, spd_diff=-650282 bps, pct=-8.1 % -------------------------------------------------------------------- avg_deviation=-7.4 %, max_deviation=-8.6 % Url=http://192.168.1.218/Data/file_1M.bin, max_speed=16000000 bps time=589 ms, dnld=1048576 B, speed=14222511 bps, spd_diff=-1777489 bps, pct=-11.1 % time=607 ms, dnld=1048576 B, speed=13816641 bps, spd_diff=-2183359 bps, pct=-13.6 % time=621 ms, dnld=1048576 B, speed=13496076 bps, spd_diff=-2503924 bps, pct=-15.6 % time=604 ms, dnld=1048576 B, speed=13870511 bps, spd_diff=-2129489 bps, pct=-13.3 % time=628 ms, dnld=1048576 B, speed=13336716 bps, spd_diff=-2663284 bps, pct=-16.6 % -------------------------------------------------------------------- avg_deviation=-14.1 %, max_deviation=-16.6 % Url=http://192.168.1.218/Data/file_1M.bin, max_speed=20000000 bps time=509 ms, dnld=1048576 B, speed=16468335 bps, spd_diff=-3531665 bps, pct=-17.7 % time=510 ms, dnld=1048576 B, speed=16430596 bps, spd_diff=-3569404 bps, pct=-17.8 % time=492 ms, dnld=1048576 B, speed=17026794 bps, spd_diff=-2973206 bps, pct=-14.9 % time=510 ms, dnld=1048576 B, speed=16429502 bps, spd_diff=-3570498 bps, pct=-17.9 % time=524 ms, dnld=1048576 B, speed=15988807 bps, spd_diff=-4011193 bps, pct=-20.1 % -------------------------------------------------------------------- avg_deviation=-17.7 %, max_deviation=-20.1 % Url=http://192.168.1.218/Data/file_1M.bin, max_speed=32000000 bps time=358 ms, dnld=1048576 B, speed=23423818 bps, spd_diff=-8576182 bps, pct=-26.8 % time=283 ms, dnld=1048576 B, speed=29537976 bps, spd_diff=-2462024 bps, pct=-7.7 % time=269 ms, dnld=1048576 B, speed=31078472 bps, spd_diff=-921528 bps, pct=-2.9 % time=286 ms, dnld=1048576 B, speed=29328541 bps, spd_diff=-2671459 bps, pct=-8.3 % time=285 ms, dnld=1048576 B, speed=29375373 bps, spd_diff=-2624627 bps, pct=-8.2 % -------------------------------------------------------------------- avg_deviation=-10.8 %, max_deviation=-26.8 % ################################################################################### Speed limit test [20 mbps], iterations=5 Url=http://192.168.1.218/Data/file_1M.bin, max_speed=8000000 bps time=1162 ms, dnld=1048576 B, speed=7214871 bps, spd_diff=-785129 bps, pct=-9.8 % time=1129 ms, dnld=1048576 B, speed=7423612 bps, spd_diff=-576388 bps, pct=-7.2 % time=1143 ms, dnld=1048576 B, speed=7338729 bps, spd_diff=-661271 bps, pct=-8.3 % time=1100 ms, dnld=1048576 B, speed=7619530 bps, spd_diff=-380470 bps, pct=-4.8 % time=1133 ms, dnld=1048576 B, speed=7403106 bps, spd_diff=-596894 bps, pct=-7.5 % -------------------------------------------------------------------- avg_deviation=-7.5 %, max_deviation=-9.8 % Url=http://192.168.1.218/Data/file_1M.bin, max_speed=16000000 bps time=616 ms, dnld=1048576 B, speed=13605302 bps, spd_diff=-2394698 bps, pct=-15.0 % time=555 ms, dnld=1048576 B, speed=15091142 bps, spd_diff=-908858 bps, pct=-5.7 % time=572 ms, dnld=1048576 B, speed=14647524 bps, spd_diff=-1352476 bps, pct=-8.5 % time=557 ms, dnld=1048576 B, speed=15049421 bps, spd_diff=-950579 bps, pct=-5.9 % time=557 ms, dnld=1048576 B, speed=15044509 bps, spd_diff=-955491 bps, pct=-6.0 % -------------------------------------------------------------------- avg_deviation=-8.2 %, max_deviation=-15.0 % Url=http://192.168.1.218/Data/file_1M.bin, max_speed=20000000 bps time=453 ms, dnld=1048576 B, speed=18511360 bps, spd_diff=-1488640 bps, pct=-7.4 % time=457 ms, dnld=1048576 B, speed=18342771 bps, spd_diff=-1657229 bps, pct=-8.3 % time=446 ms, dnld=1048576 B, speed=18783479 bps, spd_diff=-1216521 bps, pct=-6.1 % time=462 ms, dnld=1048576 B, speed=18141374 bps, spd_diff=-1858626 bps, pct=-9.3 % time=447 ms, dnld=1048576 B, speed=18758781 bps, spd_diff=-1241219 bps, pct=-6.2 % -------------------------------------------------------------------- avg_deviation=-7.5 %, max_deviation=-9.3 % -----Original Message----- From: Stefan Eissing <[email protected]> Sent: Monday, January 12, 2026 8:28 AM To: Dmitry Karpov <[email protected]> Cc: libcurl development <[email protected]> Subject: [EXTERNAL] Re: Rate limit regressions in libcurl 8.18.0 vs 8.17.0 > Am 09.01.2026 um 21:24 schrieb Dmitry Karpov <[email protected]>: > > Hi Stefan, > Thanks a lot for looking into it! > > I tried your PR, and it seems that it has improved the precision in the most > problematic cases (I tried it on 1MB and 2MB downloads), and now I don't see > any cases that look like the rate limit is not applied. > But it looks like it tends to "overthrottle", and sometimes the measured > transfer speed is below the rate limit more than ~30-50% (please see my stats > below). > > In my opinion, a small "overthrottling" (< 5%) is better than > "underthrottling", but transfer speeds that drop too low below the rate limit > may be create issues for some applications. > But hopefully, you can fine tune it, and I will be happy to help with testing. Ah, right. That does not work. I changed strategy in https://github.com/curl/curl/pull/20228 and hope that this addresses all cases better now. Would love to get your feedback. Cheers, Stefan > > Thanks! > Dmitry > > Here are the stats for 1MB and 2MB download tests for ~700 mbps, 50 mbps, 20 > mbps network speeds and 8 mbps, 16 mbps, 20 mbps and 32 mbps rate limits: > > ###################################################################### > ############# > 1 MB downloads > ###################################################################### > ############# > > Speed limit test [LAN (~700 mbps)], iterations=5 > > Url=http://192.168.1.218/Data/file_1M.bin, max_speed=8000000 bps > time=1072 ms, dnld=1048576 B, speed=7820634 bps, spd_diff=-179366 bps, > pct=-2.2 % > time=1071 ms, dnld=1048576 B, speed=7830197 bps, spd_diff=-169803 bps, > pct=-2.1 % > time=1063 ms, dnld=1048576 B, speed=7890697 bps, spd_diff=-109303 bps, > pct=-1.4 % > time=1055 ms, dnld=1048576 B, speed=7950382 bps, spd_diff=-49618 bps, > pct=-0.6 % > time=1071 ms, dnld=1048576 B, speed=7831827 bps, spd_diff=-168173 bps, > pct=-2.1 % > -------------------------------------------------------------------- > avg_deviation=-1.7 %, max_deviation=-2.2 % > > Url=http://192.168.1.218/Data/file_1M.bin, max_speed=16000000 bps > time=542 ms, dnld=1048576 B, speed=15463841 bps, spd_diff=-536159 bps, > pct=-3.4 % > time=536 ms, dnld=1048576 B, speed=15639592 bps, spd_diff=-360408 bps, > pct=-2.3 % > time=542 ms, dnld=1048576 B, speed=15464154 bps, spd_diff=-535846 bps, > pct=-3.3 % > time=542 ms, dnld=1048576 B, speed=15469744 bps, spd_diff=-530256 bps, > pct=-3.3 % > time=527 ms, dnld=1048576 B, speed=15909541 bps, spd_diff=-90459 bps, > pct=-0.6 % > -------------------------------------------------------------------- > avg_deviation=-2.6 %, max_deviation=-3.4 % > > Url=http://192.168.1.218/Data/file_1M.bin, max_speed=20000000 bps > time=439 ms, dnld=1048576 B, speed=19103703 bps, spd_diff=-896297 bps, > pct=-4.5 % > time=443 ms, dnld=1048576 B, speed=18897772 bps, spd_diff=-1102228 > bps, pct=-5.5 % > time=431 ms, dnld=1048576 B, speed=19429137 bps, spd_diff=-570863 bps, > pct=-2.9 % > time=441 ms, dnld=1048576 B, speed=19018250 bps, spd_diff=-981750 bps, > pct=-4.9 % > time=431 ms, dnld=1048576 B, speed=19433368 bps, spd_diff=-566632 bps, > pct=-2.8 % > -------------------------------------------------------------------- > avg_deviation=-4.1 %, max_deviation=-5.5 % > > Url=http://192.168.1.218/Data/file_1M.bin, max_speed=32000000 bps > time=283 ms, dnld=1048576 B, speed=29608454 bps, spd_diff=-2391546 > bps, pct=-7.5 % > time=286 ms, dnld=1048576 B, speed=29281244 bps, spd_diff=-2718756 > bps, pct=-8.5 % > time=290 ms, dnld=1048576 B, speed=28913572 bps, spd_diff=-3086428 > bps, pct=-9.6 % > time=285 ms, dnld=1048576 B, speed=29405957 bps, spd_diff=-2594043 > bps, pct=-8.1 % > time=287 ms, dnld=1048576 B, speed=29219232 bps, spd_diff=-2780768 > bps, pct=-8.7 % > -------------------------------------------------------------------- > avg_deviation=-8.5 %, max_deviation=-9.6 % > > ###################################################################### > ############# Speed limit test [50 mbps], iterations=5 > > Url=http://192.168.1.218/Data/file_1M.bin, max_speed=8000000 bps > time=1057 ms, dnld=1048576 B, speed=7929438 bps, spd_diff=-70562 bps, > pct=-0.9 % > time=1064 ms, dnld=1048576 B, speed=7882452 bps, spd_diff=-117548 bps, > pct=-1.5 % > time=1068 ms, dnld=1048576 B, speed=7850084 bps, spd_diff=-149916 bps, > pct=-1.9 % > time=1074 ms, dnld=1048576 B, speed=7806653 bps, spd_diff=-193347 bps, > pct=-2.4 % > time=1066 ms, dnld=1048576 B, speed=7866227 bps, spd_diff=-133773 bps, > pct=-1.7 % > -------------------------------------------------------------------- > avg_deviation=-1.7 %, max_deviation=-2.4 % > > Url=http://192.168.1.218/Data/file_1M.bin, max_speed=16000000 bps > time=522 ms, dnld=1048576 B, speed=16059423 bps, spd_diff=59423 bps, > pct=0.4 % > time=540 ms, dnld=1048576 B, speed=15523564 bps, spd_diff=-476436 bps, > pct=-3.0 % > time=549 ms, dnld=1048576 B, speed=15274231 bps, spd_diff=-725769 bps, > pct=-4.5 % > time=539 ms, dnld=1048576 B, speed=15544361 bps, spd_diff=-455639 bps, > pct=-2.8 % > time=539 ms, dnld=1048576 B, speed=15552027 bps, spd_diff=-447973 bps, > pct=-2.8 % > -------------------------------------------------------------------- > avg_deviation=-2.6 %, max_deviation=-4.5 % > > Url=http://192.168.1.218/Data/file_1M.bin, max_speed=20000000 bps > time=577 ms, dnld=1048576 B, speed=14525426 bps, spd_diff=-5474574 > bps, pct=-27.4 % > time=602 ms, dnld=1048576 B, speed=13927046 bps, spd_diff=-6072954 > bps, pct=-30.4 % > time=587 ms, dnld=1048576 B, speed=14268378 bps, spd_diff=-5731622 > bps, pct=-28.7 % > time=600 ms, dnld=1048576 B, speed=13957796 bps, spd_diff=-6042204 > bps, pct=-30.2 % > time=577 ms, dnld=1048576 B, speed=14527941 bps, spd_diff=-5472059 > bps, pct=-27.4 % > -------------------------------------------------------------------- > avg_deviation=-28.8 %, max_deviation=-30.4 % > > Url=http://192.168.1.218/Data/file_1M.bin, max_speed=32000000 bps > time=431 ms, dnld=1048576 B, speed=19442602 bps, spd_diff=-12557398 > bps, pct=-39.2 % > time=409 ms, dnld=1048576 B, speed=20489454 bps, spd_diff=-11510546 > bps, pct=-36.0 % > time=427 ms, dnld=1048576 B, speed=19600100 bps, spd_diff=-12399900 > bps, pct=-38.7 % > time=426 ms, dnld=1048576 B, speed=19689534 bps, spd_diff=-12310466 > bps, pct=-38.5 % > time=419 ms, dnld=1048576 B, speed=19977490 bps, spd_diff=-12022510 > bps, pct=-37.6 % > -------------------------------------------------------------------- > avg_deviation=-38.0 %, max_deviation=-39.2 % > > ###################################################################### > ############# Speed limit test [20 mbps], iterations=5 > > Url=http://192.168.1.218/Data/file_1M.bin, max_speed=8000000 bps > time=1066 ms, dnld=1048576 B, speed=7863558 bps, spd_diff=-136442 bps, > pct=-1.7 % > time=1068 ms, dnld=1048576 B, speed=7848108 bps, spd_diff=-151892 bps, > pct=-1.9 % > time=1062 ms, dnld=1048576 B, speed=7892040 bps, spd_diff=-107960 bps, > pct=-1.3 % > time=1057 ms, dnld=1048576 B, speed=7934613 bps, spd_diff=-65387 bps, > pct=-0.8 % > time=1069 ms, dnld=1048576 B, speed=7844182 bps, spd_diff=-155818 bps, > pct=-1.9 % > -------------------------------------------------------------------- > avg_deviation=-1.5 %, max_deviation=-1.9 % > > Url=http://192.168.1.218/Data/file_1M.bin, max_speed=16000000 bps > time=551 ms, dnld=1048576 B, speed=15206422 bps, spd_diff=-793578 bps, > pct=-5.0 % > time=528 ms, dnld=1048576 B, speed=15882191 bps, spd_diff=-117809 bps, > pct=-0.7 % > time=526 ms, dnld=1048576 B, speed=15942559 bps, spd_diff=-57441 bps, > pct=-0.4 % > time=542 ms, dnld=1048576 B, speed=15468688 bps, spd_diff=-531312 bps, > pct=-3.3 % > time=525 ms, dnld=1048576 B, speed=15951199 bps, spd_diff=-48801 bps, > pct=-0.3 % > -------------------------------------------------------------------- > avg_deviation=-1.9 %, max_deviation=-5.0 % > > Url=http://192.168.1.218/Data/file_1M.bin, max_speed=20000000 bps > time=776 ms, dnld=1048576 B, speed=10796719 bps, spd_diff=-9203281 > bps, pct=-46.0 % > time=792 ms, dnld=1048576 B, speed=10591462 bps, spd_diff=-9408538 > bps, pct=-47.0 % > time=773 ms, dnld=1048576 B, speed=10841258 bps, spd_diff=-9158742 > bps, pct=-45.8 % > time=793 ms, dnld=1048576 B, speed=10566860 bps, spd_diff=-9433140 > bps, pct=-47.2 % > time=790 ms, dnld=1048576 B, speed=10612754 bps, spd_diff=-9387246 > bps, pct=-46.9 % > -------------------------------------------------------------------- > avg_deviation=-46.6 %, max_deviation=-47.2 % > > Url=http://192.168.1.218/Data/file_1M.bin, max_speed=32000000 bps > time=655 ms, dnld=1048576 B, speed=12793733 bps, spd_diff=-19206267 > bps, pct=-60.0 % > time=653 ms, dnld=1048576 B, speed=12843644 bps, spd_diff=-19156356 > bps, pct=-59.9 % > time=629 ms, dnld=1048576 B, speed=13334787 bps, spd_diff=-18665213 > bps, pct=-58.3 % > time=617 ms, dnld=1048576 B, speed=13579578 bps, spd_diff=-18420422 > bps, pct=-57.6 % > time=648 ms, dnld=1048576 B, speed=12931094 bps, spd_diff=-19068906 > bps, pct=-59.6 % > -------------------------------------------------------------------- > avg_deviation=-59.1 %, max_deviation=-60.0 % > > > ###################################################################### > ############# > 2 MB downloads > ###################################################################### > ############# > > Speed limit test [LAN (~700 mbps)], iterations=5 > > Url=http://192.168.1.218/Data/file_2M.bin, max_speed=8000000 bps > time=2099 ms, dnld=2097152 B, speed=7991289 bps, spd_diff=-8711 bps, > pct=-0.1 % > time=2117 ms, dnld=2097152 B, speed=7923378 bps, spd_diff=-76622 bps, > pct=-1.0 % > time=2114 ms, dnld=2097152 B, speed=7933832 bps, spd_diff=-66168 bps, > pct=-0.8 % > time=2127 ms, dnld=2097152 B, speed=7887428 bps, spd_diff=-112572 bps, > pct=-1.4 % > time=2125 ms, dnld=2097152 B, speed=7892716 bps, spd_diff=-107284 bps, > pct=-1.3 % > -------------------------------------------------------------------- > avg_deviation=-0.9 %, max_deviation=-1.4 % > > Url=http://192.168.1.218/Data/file_2M.bin, max_speed=16000000 bps > time=1062 ms, dnld=2097152 B, speed=15793888 bps, spd_diff=-206112 > bps, pct=-1.3 % > time=1069 ms, dnld=2097152 B, speed=15680930 bps, spd_diff=-319070 > bps, pct=-2.0 % > time=1070 ms, dnld=2097152 B, speed=15674103 bps, spd_diff=-325897 > bps, pct=-2.0 % > time=1078 ms, dnld=2097152 B, speed=15550571 bps, spd_diff=-449429 > bps, pct=-2.8 % > time=1068 ms, dnld=2097152 B, speed=15694690 bps, spd_diff=-305310 > bps, pct=-1.9 % > -------------------------------------------------------------------- > avg_deviation=-2.0 %, max_deviation=-2.8 % > > Url=http://192.168.1.218/Data/file_2M.bin, max_speed=20000000 bps > time=860 ms, dnld=2097152 B, speed=19486411 bps, spd_diff=-513589 bps, > pct=-2.6 % > time=845 ms, dnld=2097152 B, speed=19837391 bps, spd_diff=-162609 bps, > pct=-0.8 % > time=839 ms, dnld=2097152 B, speed=19990582 bps, spd_diff=-9418 bps, > pct=-0.0 % > time=845 ms, dnld=2097152 B, speed=19849525 bps, spd_diff=-150475 bps, > pct=-0.8 % > time=869 ms, dnld=2097152 B, speed=19289411 bps, spd_diff=-710589 bps, > pct=-3.6 % > -------------------------------------------------------------------- > avg_deviation=-1.5 %, max_deviation=-3.6 % > > Url=http://192.168.1.218/Data/file_2M.bin, max_speed=32000000 bps > time=541 ms, dnld=2097152 B, speed=30958042 bps, spd_diff=-1041958 > bps, pct=-3.3 % > time=541 ms, dnld=2097152 B, speed=30967700 bps, spd_diff=-1032300 > bps, pct=-3.2 % > time=541 ms, dnld=2097152 B, speed=31003867 bps, spd_diff=-996133 bps, > pct=-3.1 % > time=542 ms, dnld=2097152 B, speed=30947307 bps, spd_diff=-1052693 > bps, pct=-3.3 % > time=545 ms, dnld=2097152 B, speed=30776880 bps, spd_diff=-1223120 > bps, pct=-3.8 % > -------------------------------------------------------------------- > avg_deviation=-3.3 %, max_deviation=-3.8 % > > ###################################################################### > ############# Speed limit test [50 mbps], iterations=5 > > Url=http://192.168.1.218/Data/file_2M.bin, max_speed=8000000 bps > time=2112 ms, dnld=2097152 B, speed=7942723 bps, spd_diff=-57277 bps, > pct=-0.7 % > time=2118 ms, dnld=2097152 B, speed=7918513 bps, spd_diff=-81487 bps, > pct=-1.0 % > time=2114 ms, dnld=2097152 B, speed=7935585 bps, spd_diff=-64415 bps, > pct=-0.8 % > time=2119 ms, dnld=2097152 B, speed=7915999 bps, spd_diff=-84001 bps, > pct=-1.1 % > time=2106 ms, dnld=2097152 B, speed=7963194 bps, spd_diff=-36806 bps, > pct=-0.5 % > -------------------------------------------------------------------- > avg_deviation=-0.8 %, max_deviation=-1.1 % > > Url=http://192.168.1.218/Data/file_2M.bin, max_speed=16000000 bps > time=1065 ms, dnld=2097152 B, speed=15742286 bps, spd_diff=-257714 > bps, pct=-1.6 % > time=1072 ms, dnld=2097152 B, speed=15641866 bps, spd_diff=-358134 > bps, pct=-2.2 % > time=1075 ms, dnld=2097152 B, speed=15594077 bps, spd_diff=-405923 > bps, pct=-2.5 % > time=1067 ms, dnld=2097152 B, speed=15721751 bps, spd_diff=-278249 > bps, pct=-1.7 % > time=1067 ms, dnld=2097152 B, speed=15714653 bps, spd_diff=-285347 > bps, pct=-1.8 % > -------------------------------------------------------------------- > avg_deviation=-2.0 %, max_deviation=-2.5 % > > Url=http://192.168.1.218/Data/file_2M.bin, max_speed=20000000 bps > time=935 ms, dnld=2097152 B, speed=17931176 bps, spd_diff=-2068824 > bps, pct=-10.3 % > time=939 ms, dnld=2097152 B, speed=17851539 bps, spd_diff=-2148461 > bps, pct=-10.7 % > time=927 ms, dnld=2097152 B, speed=18079130 bps, spd_diff=-1920870 > bps, pct=-9.6 % > time=922 ms, dnld=2097152 B, speed=18195737 bps, spd_diff=-1804263 > bps, pct=-9.0 % > time=935 ms, dnld=2097152 B, speed=17942471 bps, spd_diff=-2057529 > bps, pct=-10.3 % > -------------------------------------------------------------------- > avg_deviation=-10.0 %, max_deviation=-10.7 % > > Url=http://192.168.1.218/Data/file_2M.bin, max_speed=32000000 bps > time=537 ms, dnld=2097152 B, speed=31188242 bps, spd_diff=-811758 bps, > pct=-2.5 % > time=535 ms, dnld=2097152 B, speed=31324971 bps, spd_diff=-675029 bps, > pct=-2.1 % > time=524 ms, dnld=2097152 B, speed=31993531 bps, spd_diff=-6469 bps, > pct=-0.0 % > time=549 ms, dnld=2097152 B, speed=30523564 bps, spd_diff=-1476436 > bps, pct=-4.6 % > time=539 ms, dnld=2097152 B, speed=31108610 bps, spd_diff=-891390 bps, > pct=-2.8 % > -------------------------------------------------------------------- > avg_deviation=-2.4 %, max_deviation=-4.6 % > > ###################################################################### > ############# Speed limit test [20 mbps], iterations=5 > > Url=http://192.168.1.218/Data/file_2M.bin, max_speed=8000000 bps > time=2109 ms, dnld=2097152 B, speed=7951826 bps, spd_diff=-48174 bps, > pct=-0.6 % > time=2118 ms, dnld=2097152 B, speed=7920782 bps, spd_diff=-79218 bps, > pct=-1.0 % > time=2117 ms, dnld=2097152 B, speed=7923865 bps, spd_diff=-76135 bps, > pct=-1.0 % > time=2101 ms, dnld=2097152 B, speed=7983159 bps, spd_diff=-16841 bps, > pct=-0.2 % > time=2120 ms, dnld=2097152 B, speed=7912620 bps, spd_diff=-87380 bps, > pct=-1.1 % > -------------------------------------------------------------------- > avg_deviation=-0.8 %, max_deviation=-1.1 % > > Url=http://192.168.1.218/Data/file_2M.bin, max_speed=16000000 bps > time=1062 ms, dnld=2097152 B, speed=15792758 bps, spd_diff=-207242 > bps, pct=-1.3 % > time=1051 ms, dnld=2097152 B, speed=15954445 bps, spd_diff=-45555 bps, > pct=-0.3 % > time=1075 ms, dnld=2097152 B, speed=15595788 bps, spd_diff=-404212 > bps, pct=-2.5 % > time=1071 ms, dnld=2097152 B, speed=15661125 bps, spd_diff=-338875 > bps, pct=-2.1 % > time=1076 ms, dnld=2097152 B, speed=15588282 bps, spd_diff=-411718 > bps, pct=-2.6 % > -------------------------------------------------------------------- > avg_deviation=-1.8 %, max_deviation=-2.6 % > > Url=http://192.168.1.218/Data/file_2M.bin, max_speed=20000000 bps > time=1170 ms, dnld=2097152 B, speed=14327818 bps, spd_diff=-5672182 > bps, pct=-28.4 % > time=1157 ms, dnld=2097152 B, speed=14492201 bps, spd_diff=-5507799 > bps, pct=-27.5 % > time=1158 ms, dnld=2097152 B, speed=14477419 bps, spd_diff=-5522581 > bps, pct=-27.6 % > time=1146 ms, dnld=2097152 B, speed=14630854 bps, spd_diff=-5369146 > bps, pct=-26.8 % > time=1175 ms, dnld=2097152 B, speed=14269531 bps, spd_diff=-5730469 > bps, pct=-28.7 % > -------------------------------------------------------------------- > avg_deviation=-27.8 %, max_deviation=-28.7 % > > Url=http://192.168.1.218/Data/file_2M.bin, max_speed=32000000 bps > time=873 ms, dnld=2097152 B, speed=19210450 bps, spd_diff=-12789550 > bps, pct=-40.0 % > time=873 ms, dnld=2097152 B, speed=19201589 bps, spd_diff=-12798411 > bps, pct=-40.0 % > time=886 ms, dnld=2097152 B, speed=18935311 bps, spd_diff=-13064689 > bps, pct=-40.8 % > time=873 ms, dnld=2097152 B, speed=19209900 bps, spd_diff=-12790100 > bps, pct=-40.0 % > time=875 ms, dnld=2097152 B, speed=19170017 bps, spd_diff=-12829983 > bps, pct=-40.1 % > -------------------------------------------------------------------- > avg_deviation=-40.2 %, max_deviation=-40.8 % > > > -----Original Message----- > From: Stefan Eissing <[email protected]> > Sent: Friday, January 9, 2026 4:31 AM > To: libcurl development <[email protected]> > Cc: Dmitry Karpov <[email protected]> > Subject: [EXTERNAL] Re: Rate limit regressions in libcurl 8.18.0 vs > 8.17.0 > > Dimitry, > > having let this stew around in my mind the last couple of days, I came up > with a tweak that hopefully addresses your issue. > > Could you throw https://github.com/curl/curl/pull/20228 into your statistics > grinder? I would be interested to hear how it turns out. > > Thanks, > Stefan > >> Am 07.01.2026 um 21:16 schrieb Dmitry Karpov via curl-library >> <[email protected]>: >> >>> My point is that just the same statement: "there are cases that look like >>> as the rate limit hasn’t applied at all" - is also true for 8.6.0. The only >>> difference is that with 8.6.0 you have to go to smaller transfer sizes to >>> see the effect. >> >> No, this is not true. As I mentioned in my message, in 8.6.0, the smaller >> transfer sizes decreased accuracy, but it was always applied unless the >> transfers were extremely small. >> But in 8.18.0 we have cases when no rate limit is applied to the transfers >> depending on run-time conditions. >> >>> So yes, at a given transfer size the behavior may be different, because the >>> rate-limiting "resolution" has changed, but claiming that this is a radical >>> change in behavior seems wrong. >> >> It is a radical change, because we now have cases in 8.18.0, when the rate >> limit is either not applied at all or there is a huge gap between the rate >> limit and the measured transfer speed. >> And it may increase the operational costs significantly for libcurl users in >> such cases. >> >> Let’s consider an example from my test where the rate limit wasn’t applied: >> Url=… file_1M.bin, max_speed=20000000 bps >> time=18 ms, dnld=1048576 B, speed=457518843 bps, spd_diff=437518843 >> bps, pct=2187.6 % >> >> If I do multiple downloads like this back-to-back (i.e. 100+ times) then CPU >> utilization for these transfers will be as high as without the rate limit >> both for client and the server. >> But in 8.6.0 – where the rate limit for these conditions was applied >> correctly, the total CPU utilization (both for client and server) was a few >> times lower. >> >> So, while 8.18.0 rate limiting mechanism decreased CPU utilization >> compared to 8.6.0 for the cases where it works, it increased CPU utilization >> on order of magnitude both for client and server for run-time conditions >> where it doesn’t work. And this may have a large cost effect for >> applications with large population using cloud services. >> >> And what makes things worse is that it is now very difficult to predict >> where the new mechanism will “not work”. >> >> In the same test example, the rate limit of 16 Mbps was applied correctly: >> Url=… file_1M.bin, max_speed=16000000 bps >> time=510 ms, dnld=1048576 B, speed=16423165 bps, spd_diff=423165 bps, >> pct=2.6 % >> >> This means that in 8.18.0 some client app doing multiple back-to-back >> transfers, may suddenly step on cases where the rate limiting doesn’t work >> as before and thus incur much higher operational costs. >> And this kind of defeats the purpose of optimizing the rate limiting >> mechanism to use less CPU cycles. >> >> I am trying to avoid calling it “regression”, but it is a radical behavior >> change, which potentially can cost much more to libcurl users than previous >> implementation. >> >> Thanks, >> Dmitry >> From: David Pfitzner <[email protected]> >> Sent: Tuesday, January 6, 2026 7:36 PM >> To: Dmitry Karpov <[email protected]> >> Cc: libcurl development <[email protected]> >> Subject: Re: [EXTERNAL] Re: Rate limit regressions in libcurl 8.18.0 vs >> 8.17.0 >> On Wed, Jan 7, 2026 at 1:02 PM Dmitry Karpov <[email protected]> wrote: >> In the 8.6.0, the rate limiter mechanism was kind of embedded into the run >> loop, so it was always engaged and showed very good precision (< 5% >> deviation of the measured speed from the rate limit) even for relatively >> small transfer sizes (~300 KB) and super-fast network speeds. >> >> It worked properly even when client and server were on the same host machine >> where there were no network delays. >> Here is the test example for 390 KB download throttling test on 8.6.0 when >> server and client are on the same machine: >> (8.6.0) Speed limit test [server on the same host as client], >> iterations=5 Url=http://192.168.1.3/Data/DSC_391kB.jpg, >> max_speed=8000000 bps >> time=399 ms, dnld=400096 B, speed=8020969 bps, spd_diff=20969 bps, >> pct=0.3 % >> time=405 ms, dnld=400096 B, speed=7884597 bps, spd_diff=-115403 bps, >> pct=-1.4 % >> time=406 ms, dnld=400096 B, speed=7882480 bps, spd_diff=-117520 bps, >> pct=-1.5 % >> time=404 ms, dnld=400096 B, speed=7910005 bps, spd_diff=-89995 bps, >> pct=-1.1 % >> time=397 ms, dnld=400096 B, speed=8056036 bps, spd_diff=56036 bps, >> pct=0.7 % >> -------------------------------------------------------------------- >> avg_deviation=-0.6 %, max_deviation=-1.5 % >> Url=http://192.168.1.3/Data/DSC_391kB.jpg, max_speed=16000000 bps >> time=209 ms, dnld=400096 B, speed=15286666 bps, spd_diff=-713334 bps, >> pct=-4.5 % >> time=197 ms, dnld=400096 B, speed=16231980 bps, spd_diff=231980 bps, >> pct=1.4 % >> time=197 ms, dnld=400096 B, speed=16202236 bps, spd_diff=202236 bps, >> pct=1.3 % >> time=198 ms, dnld=400096 B, speed=16110410 bps, spd_diff=110410 bps, >> pct=0.7 % >> time=195 ms, dnld=400096 B, speed=16394185 bps, spd_diff=394185 bps, >> pct=2.5 % >> -------------------------------------------------------------------- >> avg_deviation=0.3 %, max_deviation=-4.5 % >> Url=http://192.168.1.3/Data/DSC_391kB.jpg, max_speed=20000000 bps >> time=162 ms, dnld=400096 B, speed=19697882 bps, spd_diff=-302118 bps, >> pct=-1.5 % >> time=161 ms, dnld=400096 B, speed=19856004 bps, spd_diff=-143996 bps, >> pct=-0.7 % >> time=166 ms, dnld=400096 B, speed=19249034 bps, spd_diff=-750966 bps, >> pct=-3.8 % >> time=162 ms, dnld=400096 B, speed=19721549 bps, spd_diff=-278451 bps, >> pct=-1.4 % >> time=159 ms, dnld=400096 B, speed=20035479 bps, spd_diff=35479 bps, >> pct=0.2 % >> -------------------------------------------------------------------- >> avg_deviation=-1.4 %, max_deviation=-3.8 % >> Url=http://192.168.1.3/Data/DSC_391kB.jpg, max_speed=32000000 bps >> time=97 ms, dnld=400096 B, speed=32785684 bps, spd_diff=785684 bps, >> pct=2.5 % >> time=99 ms, dnld=400096 B, speed=32143324 bps, spd_diff=143324 bps, >> pct=0.4 % >> time=99 ms, dnld=400096 B, speed=32290545 bps, spd_diff=290545 bps, >> pct=0.9 % >> time=98 ms, dnld=400096 B, speed=32509298 bps, spd_diff=509298 bps, >> pct=1.6 % >> time=98 ms, dnld=400096 B, speed=32478290 bps, spd_diff=478290 bps, >> pct=1.5 % >> -------------------------------------------------------------------- >> avg_deviation=1.4 %, max_deviation=2.5 % >> >> The precision started deteriorating (> 15%) only for the downloads < >> 250 KB, but the measured transfer speeds were always less than the rate >> limit, so, the back-to-back transfers never created high load issues on the >> server side. >> >> So, to be on the safe side, my statement about high precision of rate >> limiting in 8.6.0 should be corrected as "for any transfer sizes > 1MB and >> any network conditions". >> >> Unfortunately, the price for such a precision in 8.6.0 was higher CPU >> utilization as more cycles were needed to calculate and apply delays. >> >> So, the intent to minimize CPU usage while losing some precision in 8.18 is >> totally understandable, but the loss of precision due to lesser points of >> delay calculations should be somehow compensated at the end of the transfer >> to avoid regressions for applications that expect more or less reliable >> correlation between applied rate limit and measured transfer speeds. >> >> Unfortunately, in 8.18.0 there are cases that look like as the rate >> limit hasn’t applied at all, and it creates issues for logic making >> decisions based on transfer speed measurements, and unexpected higher load >> on the server side. >> My point is that just the same statement: "there are cases that look like as >> the rate limit hasn’t applied at all" - is also true for 8.6.0. The only >> difference is that with 8.6.0 you have to go to smaller transfer sizes to >> see the effect. So it depends on what transfer sizes you are interested in >> (and other parameters). So yes, at a given transfer size the behavior may be >> different, because the rate-limiting "resolution" has changed, but claiming >> that this is a radical change in behavior seems wrong. >> -- David >> >> >> From: curl-library <[email protected]> On Behalf Of >> David Pfitzner via curl-library >> Sent: Tuesday, January 6, 2026 5:17 PM >> To: libcurl development <[email protected]> >> Cc: David Pfitzner <[email protected]> >> Subject: [EXTERNAL] Re: Rate limit regressions in libcurl 8.18.0 vs 8.17.0 >> On Wed, Jan 7, 2026 at 10:05 AM Dmitry Karpov via curl-library >> <[email protected]> wrote: >> >> The 8.6.0 worked very well for rate limiting as it provided stable and >> predictable measured transfer download speeds for any transfer sizes and >> network conditions. >> That "for any transfer sizes and network conditions" is not true. The rate >> limiting may have done what you wanted for the parameters you care about, or >> the parameters that you happened to test, but even for 8.6.0 there are >> possible cases where rate limiting is specified but has no effect. (That is, >> because the transfer finishes before any rate limiting delay is imposed.). >> -- David >> -- >> Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library >> Etiquette: https://curl.se/mail/etiquette.html > > -- Unsubscribe: https://lists.haxx.se/mailman/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
