Greetings Neal, On Monday 17 February 2003 06:41, Neal Richter wrote: > First question: What were the results with the pagesize @ 32K??
The run I didn't didn't produce any errors, but that doesn't give me much confidence. The problem is elusive and I haven't yet played around with the data set (using exclude_urls) to try to reproduce it, as I have had to with 8k. > If you are indexing the same data in the same order every time > and the target moves with trivial changes of code or flags.. that > usually means MEMORY CORRUPTION!!! > Unfortunately these types of errors are difficult to duplicate on > other machines/platforms. True. But is the order actually the same? I haven't read the code thoroughly, but I thought ht://Dig had a timing mechanism in order not to flood a particular server. Couldn't that reorder the search depending on how much time is spent on debugging output? > Valgrind is a nice open source memory debugger. > This will definetly help find a memory error, but will also drown > you in output sice htdig is pretty bad with memory leaks.. but you > can dissable leak detection and look only for memory corruption. Thanks for the tip. I had used mpatrol previously, which seems more powerful, with the corresponding increase in hassle. Interestingly, it didn't report any leaks or reproduce the error... I'm rerunning it without valgrind now to check that the error is actually repeatable. > I'd also like a link to your test data if you can put it up on a > high bandwidth server.. and a copy of your conf file and > commandline htdig options. I'd like to run Insure++ on it. I'll see what I can do. However, since all the URLs are file:///s, the actual data set will differ for you, unless you overwrite your current filesystem. Remember, this is a *very* pernickety bug! Now that you have suggested some tools, I'll keep playing around until I have something more concrete to report. Thanks, as always, Lachlan ------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ htdig-dev mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/htdig-dev
