Thanks, Joe :) There are some unexpected things about this profile. In particular, it seems that some files are not being profiled.
* regcomp is taking *much* too much time! As I understand, regular expressions are being used for things which really "should" be just straight string compares. I vote that we go back to doing standard compares rather than create an escaped string and then call an expensive regex function... * Rather than 'fork'ing a separate external parser each time, could we look at having a "persistent parser", like a persistent TCP connection? It would require a way of specifying the end of one file and the start of the next, but it looks like the performance gain might be worthwhile. * regcomp, calloc and gethostbyname all seem to be being called a lot more often than gprof recognises. * gethostbyname seems to be too expensive. Joe, were you using "persistent connections"? Gabriele, could we reduce the number of times gethostbyname is called? Perhaps we could cache the names? * None of the functions in Connection.cc seem to be profiled. Any ideas why? Thanks all, Lachlan On Wed, 14 Apr 2004 14:36, Joe R. Jah wrote: > I compiled htdig-3.2.0b5 with -pg; the following patches applied: > DESTDIR.0 TMPFILE.0 extension_filter.0 fileSpace.0 operator[].0 and > robots.0 > > I ran htdig on ~13k documents; it ran about 40% slower than my > regular htdig, (without -pg). I ran gprof htdig > htdig.gmon;gzip > htdig.gmon, and put the profile on the patch site, although it's > not a patch;) > > ftp://ftp.ccsf.org/htdig-patches/3.2.0b5/htdig.gmon.gz > > Hope it can help in improving htdig performance. > > Regards, > > Joe -- [EMAIL PROTECTED] ht://Dig developer DownUnder (http://www.htdig.org) ------------------------------------------------------- This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id70&alloc_id638&op=click _______________________________________________ ht://Dig Developer mailing list: [EMAIL PROTECTED] List information (subscribe/unsubscribe, etc.) https://lists.sourceforge.net/lists/listinfo/htdig-dev
