Is this file to be used with 3.1.9 sources, or 3.1.10? (Either is fine - I can adjust
as necessary quite easily).
Thanks for the fix. I have over a million urls inserted and climbing. :-)
-- Dan
On Thu, 15 Feb 2001, Alexander Barkov wrote:
> Dan,please take new cache.c and recompile everything.
> It should fix the problem.
>
>
> [EMAIL PROTECTED] wrote:
> >
> > I just have to put in my encounters here, because they seem very similar. I get a
>large amount of information indexed, but upon trying to run splitter, it will core
>dump somewhere midway through, and on one round left wierd directories in the
>$VAR/raw directory:
> >
> > [root@spider raw]# ls -al
> > total 32988
> > drwxr-xr-x 5 root root 8192 Feb 14 04:13 .
> > drwxr-xr-x 6 root root 4096 Feb 13 01:58 ..
> > drwxr-xr-x 3 root root 4096 Feb 13 03:12 64
> > -rw------- 1 root root 33132544 Feb 14 04:13 core
> > -rw-r--r-- 1 root root 8464 Feb 14 04:22 del.log
> > -rw-r--r-- 1 root root 566272 Feb 14 04:22 wrd.log
> > drwxr-xr-x 3 root root 4096 Feb 13 03:58 ?Ë?64
> > drwxr-xr-x 3 root root 4096 Feb 13 06:06 à??18
> > [root@spider raw]#
> >
> > Unfortunately I wasn't thinking and I deleted all the .done files, and not all of
>the logs were split. Well, back to indexing...
> >
> > I'm using 3.1.9 on Linux/Oracle
> >
> > -- Dan Hanks
> >
> > On Wed, 14 Feb 2001, Zenon Panoussis wrote:
> >
> > >
> > >
> > > Zenon Panoussis skrev:
> > > >
> > >
> > > > By now, I have almost 1 GB of indexed files, 4 indexer
> > > > crashes and one splitter crash. I'll do the debugging and
> > > > post its output tomorrow.
> > >
> > > ===================
> > > # gdb indexer core.indexer.01
> > > GNU gdb 5.0
> > > Copyright 2000 Free Software Foundation, Inc.
> > > GDB is free software, covered by the GNU General Public License, and you
> > > are
> > > welcome to change it and/or distribute copies of it under certain
> > > conditions.
> > > Type "show copying" to see the conditions.
> > > There is absolutely no warranty for GDB. Type "show warranty" for
> > > details.
> > > This GDB was configured as "i386-redhat-linux"...
> > > Core was generated by `./indexer -m -s 200'.
> > > Program terminated with signal 11, Segmentation fault.
> > > Reading symbols from /usr/lib/mysql/libmysqlclient.so.10...done.
> > > Loaded symbols for /usr/lib/mysql/libmysqlclient.so.10
> > > Reading symbols from /lib/libm.so.6...done.
> > > Loaded symbols for /lib/libm.so.6
> > > Reading symbols from /usr/lib/libz.so.1...done.
> > > Loaded symbols for /usr/lib/libz.so.1
> > > Reading symbols from /lib/libc.so.6...done.
> > > Loaded symbols for /lib/libc.so.6
> > > Reading symbols from /lib/libcrypt.so.1...done.
> > > Loaded symbols for /lib/libcrypt.so.1
> > > Reading symbols from /lib/libnsl.so.1...done.
> > > Loaded symbols for /lib/libnsl.so.1
> > > Reading symbols from /lib/ld-linux.so.2...done.
> > > Loaded symbols for /lib/ld-linux.so.2
> > > Reading symbols from /lib/libnss_files.so.2...done.
> > > Loaded symbols for /lib/libnss_files.so.2
> > > Reading symbols from /lib/libnss_nisplus.so.2...done.
> > > Loaded symbols for /lib/libnss_nisplus.so.2
> > > Reading symbols from /lib/libnss_nis.so.2...done.
> > > Loaded symbols for /lib/libnss_nis.so.2
> > > Reading symbols from /lib/libnss_dns.so.2...done.
> > > Loaded symbols for /lib/libnss_dns.so.2
> > > Reading symbols from /lib/libresolv.so.2...done.
> > > Loaded symbols for /lib/libresolv.so.2
> > > #0 0x805e5fa in UdmCRC32 (buf=0x4021b03e "", size=4294967295) at
> > > crc32.c:97
> > > 97 _CRC32_(crc, *p) ;
> > > (gdb) print crc
> > > $1 = 1928826335
> > > (gdb) print p
> > > $2 = 0x40431000 <Address 0x40431000 out of bounds>
> > >
> > > ===================
> > >
> > > # gdb indexer core.indexer.02
> > > <snip loading>
> > > #0 0x805e5fa in UdmCRC32 (buf=0x4021b03e "", size=4294967295) at
> > > crc32.c:97
> > > 97 _CRC32_(crc, *p) ;
> > > (gdb) print crc
> > > $1 = 835566978
> > > (gdb) print p
> > > $2 = 0x40404000 <Address 0x40404000 out of bounds>
> > >
> > > ===================
> > >
> > > # gdb indexer core.indexer.03
> > > <snip loading>
> > > #0 0x805e5fa in UdmCRC32 (buf=0x4021b03e "", size=4294967295) at
> > > crc32.c:97
> > > 97 _CRC32_(crc, *p) ;
> > > (gdb) print crc
> > > $1 = 2869617068
> > > (gdb) print p
> > > $2 = 0x40404000 <Address 0x40404000 out of bounds>
> > >
> > > ===================
> > >
> > > # gdb indexer core.indexer.04
> > > <snip loading>
> > > (gdb) print crc
> > > $1 = 1253677059
> > > (gdb) print p
> > > $2 = 0x40431000 <Address 0x40431000 out of bounds>
> > >
> > > ===================
> > >
> > > And finally the splitter:
> > >
> > > # gdb splitter core.splitter.01
> > > <snip copyright>
> > > This GDB was configured as "i386-redhat-linux"...
> > > Core was generated by `/usr/local/mnogo3110/sbin/splitter'.
> > > Program terminated with signal 11, Segmentation fault.
> > > Reading symbols from /usr/lib/mysql/libmysqlclient.so.10...done.
> > > Loaded symbols for /usr/lib/mysql/libmysqlclient.so.10
> > > Reading symbols from /lib/libm.so.6...done.
> > > Loaded symbols for /lib/libm.so.6
> > > Reading symbols from /usr/lib/libz.so.1...done.
> > > Loaded symbols for /usr/lib/libz.so.1
> > > Reading symbols from /lib/libc.so.6...done.
> > > Loaded symbols for /lib/libc.so.6
> > > Reading symbols from /lib/libcrypt.so.1...done.
> > > Loaded symbols for /lib/libcrypt.so.1
> > > Reading symbols from /lib/libnsl.so.1...done.
> > > Loaded symbols for /lib/libnsl.so.1
> > > Reading symbols from /lib/ld-linux.so.2...done.
> > > Loaded symbols for /lib/ld-linux.so.2
> > > #0 0x8057d15 in UdmSplitCacheLog (log=118) at cache.c:635
> > > 635
> > > logwords[count+j].wrd_id=table[w].wrd_id;
> > > (gdb) print count
> > > $1 = 13121220
> > > (gdb) print count+j
> > > $2 = 13125316
> > > (gdb) print logwords
> > > $3 = (UDM_LOGWORD *) 0x0
> > > (gdb) print table[w]
> > > $4 = {wrd_id = 1918989871, weight = 1869507887, pos = 825454439, len =
> > > 1949249585}
> > > (gdb) print logwords[count+j]
> > > Cannot access memory at address 0x15e7bd70
> > >
> > > ===================
> > >
> > > This time I'm keeping the core dumps, so let me know if there's
> > > anything else you want me to check.
> > >
> > > Apart from this, I got some garbage directories with misnamed
> > > splitter files in them in sbin:
> > >
> > > # pwd
> > > /usr/local/mnogo3110/sbin
> > > # ls -l
> > > <snip normal stuff>
> > > -rw-r--r-- 1 root root 457672 Feb 13 08:28 àË???
> > > drwxr-xr-x 3 root root 4096 Feb 13 08:28 àË???3F
> > > -rw-r--r-- 1 root root 487224 Feb 13 08:27 æmE<?56.tmp
> > >
> > > # ls -lR
> > > .:
> > > total 403708
> > > -rwxr-xr-x 1 root root 67163 Feb 12 18:02 cachelogd
> > > -rw-r--r-- 1 root root 0 Feb 14 06:18 cachelogd.out
> > > -rw------- 1 root root 3862528 Feb 13 02:39 core.indexer.01
> > > -rw------- 1 root root 3416064 Feb 13 06:06 core.indexer.02
> > > -rw------- 1 root root 2953216 Feb 13 06:57 core.indexer.03
> > > -rw------- 1 root root 3235840 Feb 13 07:51 core.indexer.04
> > > -rw------- 1 root root 397484032 Feb 14 06:32
> > > core.splitter.01
> > > -rwxr-xr-x 1 root root 513960 Feb 12 18:02 indexer
> > > -rwxr-xr-x 1 root root 1502 Feb 12 20:28 run-splitter
> > > -rwxr-xr-x 1 root root 472624 Feb 12 18:02 splitter
> > > -rw-r--r-- 1 root root 457672 Feb 13 08:28 àË???
> > > drwxr-xr-x 3 root root 4096 Feb 13 08:28 àË???3F
> > > -rw-r--r-- 1 root root 487224 Feb 13 08:27 æmE<?56.tmp
> > >
> > > ./àË???3F:
> > > total 4
> > > drwxr-xr-x 2 root root 4096 Feb 13 08:28 1
> > >
> > > ./àË???3F/1:
> > > total 80
> > > -rw-r--r-- 1 root root 312 Feb 13 08:28 3F1AD000
> > > -rw-r--r-- 1 root root 40 Feb 13 08:28 3F1B0000
> > > -rw-r--r-- 1 root root 536 Feb 13 08:28 3F1B3000
> > > -rw-r--r-- 1 root root 184 Feb 13 08:28 3F1B4000
> > > -rw-r--r-- 1 root root 88 Feb 13 08:28 3F1B8000
> > > -rw-r--r-- 1 root root 168 Feb 13 08:28 3F1C0000
> > > -rw-r--r-- 1 root root 40 Feb 13 08:28 3F1C5000
> > > -rw-r--r-- 1 root root 136 Feb 13 08:28 3F1C9000
> > > -rw-r--r-- 1 root root 248 Feb 13 08:28 3F1CC000
> > > -rw-r--r-- 1 root root 56 Feb 13 08:28 3F1CE000
> > > -rw-r--r-- 1 root root 168 Feb 13 08:28 3F1D3000
> > > -rw-r--r-- 1 root root 56 Feb 13 08:28 3F1D5000
> > > -rw-r--r-- 1 root root 56 Feb 13 08:28 3F1D6000
> > > -rw-r--r-- 1 root root 40 Feb 13 08:28 3F1D8000
> > > -rw-r--r-- 1 root root 40 Feb 13 08:28 3F1DA000
> > > -rw-r--r-- 1 root root 40 Feb 13 08:28 3F1DE000
> > > -rw-r--r-- 1 root root 40 Feb 13 08:28 3F1E4000
> > > -rw-r--r-- 1 root root 104 Feb 13 08:28 3F1E7000
> > > -rw-r--r-- 1 root root 40 Feb 13 08:28 3F1E9000
> > > -rw-r--r-- 1 root root 72 Feb 13 08:28 3F1F8000
> > >
> > > I made a tar.gz of this, in case you want to look at it. It's at
> > > http://search.freewinds.cx/garbage_in_sbin.tar.gz
> > >
> > > Z
> > >
> > >
> > >
> >
> > --
> > /========================================================================\
> > || Daniel Hanks - Systems/Database Administrator ||
> > || About Inc., Web Services Division ||
> > || 1253 N. Research Way, Suite Q-2500. Orem, UT 84097 ||
> > || ph: 801-437-6023 fax: 801-437-6020 email: [EMAIL PROTECTED] ||
> > \========================================================================/
> >
> > ______________
> > If you want to unsubscribe send "unsubscribe udmsearch"
> > to [EMAIL PROTECTED]
>
>
--
/========================================================================\
|| Daniel Hanks - Systems/Database Administrator ||
|| About Inc., Web Services Division ||
|| 1253 N. Research Way, Suite Q-2500. Orem, UT 84097 ||
|| ph: 801-437-6023 fax: 801-437-6020 email: [EMAIL PROTECTED] ||
\========================================================================/
______________
If you want to unsubscribe send "unsubscribe udmsearch"
to [EMAIL PROTECTED]