On Tuesday 03 November 2015 12:11:28 Lauri Kasanen wrote: > > Thanks for alerting us about this! > > > > I found this case so intersting I ran my own test. I ran the function > > over 500MB of data from /dev/urandom and on this test data I could see > > a slightly better than 40% performance gain (with gcc -O3 on my 3.5GHz > > Intel i7 CPU). > > > > Still: "This was causing a two-second startup delay" > > seems unlikely. I could run the unoptimized version through 500MB of > > data in 1.5 seconds, and surely any sensible cookie use cases would be > > several magnitudes away of that? > > Hi, > > I have less than 1mb of cookies, 9203 according to wc -l. The timing > wasn't scientifically exact, but my senses aren't that far off to > mistake 0.something to 2 ;) > > When I ran oprofile on just the app startup, curl was using 90% of cpu, > most of it in Curl_add_cookie and Curl_raw_equal. After this change, > Curl_raw_equal dropped from 55% to 29%. (function names from memory) > > I could also separate the delay to curl by starting to an empty tab > instead of a web page: the delay would appear as soon as I wrote an > address and pressed enter, ie at the first invocation of curl code. > > Now with this change, the startup delay has approximately halved, to > about one sec. I'm running on a Phenom 2. I didn't delve deep into the > cookie code, but it seems every cookie's every field is matched against > every other's on a quick look. 10k ^ 2 = 100M checks, times the number > of fields.
Curl uses a linked list for cookies. That's fine when you have just a bunch of entries. Adding an entry takes O(n^2). Using a hash table would reduce it to O(n). In other words: if 10k entries now take 1s, with a hashtable it would be (best case) 0.1ms. With a bunch of overhead per entry (a constant factor to be multiplied to the 0.1ms), let's say it is 1ms for 10k entries. Curl has a hash API, so it should be straight forward to improve the cookie code. Tim ------------------------------------------------------------------- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
