According to Sean Downey: > another question > I've been having problems when we htmerge - I think it's a memory problem > > I get a message "Abort trap (cored dumped)" > > I looked up the mailing list archives and some other guy had a similar > problem > http://sourceforge.net/mailarchive/message.php?msg_id=220767 > he said he hacked the words.cc file which you said was needed for BSD > systems. > > I couldn't find the words.cc file > I'm using snapshot htdig-3.2.0b4-20020303 > > is there another file I can change to modify the default tmp directory
Hi, Sean. In the message you reference above, I was pointing out that Nico and Mike had two different problems. Nico had problems with htdig causing the whole system to run out of memory and kill off other processes, while Mike had a problem with htmerge filling up the /tmp volume. However, both of these problems were with htdig/htmerge 3.1.x. You're using a 3.2 beta release, which handles things very differently. In 3.2, htmerge is contained mostly in one source file (httools/htmerge.cc) so there's no words.cc file anymore. Also, htmerge doesn't use /tmp at all in 3.2, because it doesn't use an external sort program to deal with an ASCII wordlist file, but instead deals with the word databases directly. So, you won't get any useful/applicable advice from that archived message. There is a known problem with htmerge in 3.2, though. It's a real memory hog, so it does tend to have problems when merging large databases. I don't know what the main causes of its inefficient use of memory are, so I'm at a bit of a loss to suggest any fixes to it at this point. One thing I do know is that it tries to store a complete list of URLs in memory from the database to be merged, and that can chew up a lot of RAM. It would probably help if it were rewritten to use a list of DocIDs instead, the way htpurge and htnotify work. I'm sure that would cut down on memory usage, but I don't know enough about the inner workings of the database code to say for sure whether this would be the fix to the problem or if there are other places that chew up excessive RAM. Having said that, though, it's not clear to me whether the cause of the "Abort trap" you're getting is that htmerge is running out of memory, or if it's caused by something else (e.g. database corruption, bugs in the DB code). It might help to get a stack backtrace from the core dump, e.g. "gdb /path/to/htmerge /path/to/core" then "bt", assuming gdb is the debugger available on your FreeBSD system. -- Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930 _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
