You don't need a recursive download; any https URL does this. This is all you need to show the problem: $ /usr/bin/time wget -O /dev/null https://google.com/
13.79user 0.49system 0:14.60elapsed 97%CPU (0avgtext+0avgdatan 1800320maxresident)k 0inputs+0outputs (0major+113179minor)pagefaults 0swaps All of this time and memory is spent on parsing SSL certificates, which you can see with $ strace -tt -f -e open wget -O /dev/null https://www.google.com/ It seems like it's due to gnutls being slow. This blog post describes some work that was done to improve that -- I wonder if it was ever pushed upstream? http://blog.josefsson.org/2008/02/27/real-world-performance-tuning-with-callgrind/ -jim -- To UNSUBSCRIBE, email to [email protected] with a subject of "unsubscribe". Trouble? Contact [email protected]
