---------- Forwarded message ----------
Date: Wed, 26 Sep 2001 00:19:55 +0200 (IST)
From: Peter L. Peres <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: SIGSEGV on large index with 3.1.5
Hi,
I got a repeatable SISGSEGV on htdig -ias on a single machine with Apache
web server (htdig indexes through Apache not directly). The problem
occured three times so far, but it takes several hours to manifest itself
and I only have a core file from the last run. gdb-ing this yields:
------------
This GDB was configured as "i686-pc-linux-gnu"...
Core was generated by `/opt/www/htdig/bin/htdig -ias -v -c
/opt/www/htdig/conf/htdig.conf'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/lib/libz.so.1...done.
Reading symbols from /usr/lib/libstdc++.so.2.9...done.
Reading symbols from /lib/libm.so.6...done.
Reading symbols from /lib/libc.so.6...done.
Reading symbols from /lib/ld-linux.so.2...done.
Reading symbols from /lib/libnss_files.so.1...done.
#0 Dictionary::Find (this=0x80b2560, name=0x82d5020 "4367950")
at Dictionary.cc:242
242 if (e->hash == hash && strcmp(e->key, name) == 0)
(gdb) p e
$1 = (DictionaryEntry *) 0x40
(gdb) p hash
$2 = 4367950
(gdb) p name
$3 = 0x82d5020 "4367950"
(gdb) p e->key
$4 = 0x61692d20 <Address 0x61692d20 out of bounds>
(gdb) p table
$5 = (DictionaryEntry **) 0x61692d20
(gdb) p index
$6 = {<text variable, no debug info>} 0x40007660 <index>
(gdb) p table[index]
Argument to arithmetic operation not a number or boolean.
// huh ? Never seen that gdb error before.
-----------
Apparently e is set to something illegal. I did not have the time to look
at the code. Has anyone seen something like this ? The database size is
about 400,000 documents, some html, many source and text files, totaling
about 700MB (index.db) when completed. The problem occured when the size
of the data to be indexed increased by another 10% or so (50MB give or
take).
I have not tried but it is possible that by splitting the database into
smaller realms and indexing each alone would avoid the bug (is it a bug
?).
Has anyone seen this or a similar problem and if yes, then under what
circumstances ? Bad hardware etc is almost completely ruled out, the
machine works well all the time, but there is always a possibility for
something to go wrong.
Apparently whatever builds lists of hashes and inserts into table[] has a
bug (I had no time to look). I will keep this core file so you can ask
more questions if you want to. The core file size is:
137736192 Sep 22 12:47 core
(~138MB) fyi. It took me this long to get around to write this message. It
is just barely possible that I ran out of virtual memory but I don't think
that this was the problem. The e pointer value is clearly invalid. I have
a total of 220MB VM available for now. I can increase this. Is there a
guide as to how much memory is required to index a document tree how big ?
E.g. do I need 700MB VM to index 700MB of documents ? Half that much ?
(350MB) ? Shouldn't the error occur when new() is called to generate the
new pointer instead of giving a SIGSEGV ?
thank you in advance for any insights,
Peter
[EMAIL PROTECTED]
PS: PLEASE CC MAIL, I AM NOT ON THE LIST NOW.
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a
subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
