Hello!
We finally found a bug in cache.c. New version is in attachement.
Everybody who has problems with splitter's crashes are welcome to test.
Please, give feedback!
#include "udm_config.h"
#include stdio.h
#include stdlib.h
#include string.h
#include errno.h
#ifdef HAVE_UNISTD_H
#include
Caffeinate The World wrote:
--- Alexander Barkov [EMAIL PROTECTED] wrote:
Caffeinate The World wrote:
--- Alexander Barkov [EMAIL PROTECTED] wrote:
Hello!
We finally found a bug in cache.c. New version is in attachement.
Everybody who has problems with splitter's
--- Alexander Barkov [EMAIL PROTECTED] wrote:
Caffeinate The World wrote:
--- Alexander Barkov [EMAIL PROTECTED] wrote:
Hello!
We finally found a bug in cache.c. New version is in attachement.
Everybody who has problems with splitter's crashes are welcome to
test.
There was actually two bugs. The first one because of table[4096]
in sql.c. Sometimes it may became more large. Now dinamic realloc'ing
has been added.
Opps. I mean in cache.c, not sql.c
The second one that cmpcache() function passed to qsort gave ordering
slightly different with
Alexander Barkov skrev:
We finally found a bug in cache.c. New version is in attachement.
Everybody who has problems with splitter's crashes are welcome to test.
Please, give feedback!
You guys are great! I'll re-compile and get back to you with
reports.
BTW, can I remove
Zenon Panoussis wrote:
Alexander Barkov skrev:
We finally found a bug in cache.c. New version is in attachement.
Everybody who has problems with splitter's crashes are welcome to test.
Please, give feedback!
You guys are great! I'll re-compile and get back to you with
reports.
--- Alexander Barkov [EMAIL PROTECTED] wrote:
Hello!
We finally found a bug in cache.c. New version is in attachement.
Everybody who has problems with splitter's crashes are welcome to
test.
should the 'tree' directory be removed? can we split the raw log files
we have thus far or is
Alexander Barkov skrev:
We finally found a bug in cache.c. New version is in attachement.
Everybody who has problems with splitter's crashes are welcome to test.
Please, give feedback!
Oops. Something else is not OK:
cache.c:687:87: warning: #ifdef with no argument
cache.c:692:87:
Zenon Panoussis skrev:
Oops. Something else is not OK:
cache.c:687:87: warning: #ifdef with no argument
[etc]
I think that the mailer is responsible for this. There are
lots of broken lines in the code, that shouldn't be broken.
Perhaps it's better to attach the file in .gz format
i didn't get this error on my NetBSD/Alpha. compile was fine.
what system are you on?
--- Zenon Panoussis [EMAIL PROTECTED] wrote:
Alexander Barkov skrev:
We finally found a bug in cache.c. New version is in attachement.
Everybody who has problems with splitter's crashes are
Is this file to be used with 3.1.9 sources, or 3.1.10? (Either is fine - I can adjust
as necessary quite easily).
Thanks for the fix. I have over a million urls inserted and climbing. :-)
-- Dan
On Thu, 15 Feb 2001, Alexander Barkov wrote:
Dan,please take new cache.c and recompile
Caffeinate The World skrev:
i've been going through this and back again time and time again. what
would really be nice is indexer save the logs in a format that's easy
to use again. for instance, you can use the format re-index to sql etc.
or if you want to reindex again, you don't
Zenon Panoussis wrote:
Caffeinate The World skrev:
i've been going through this and back again time and time again. what
would really be nice is indexer save the logs in a format that's easy
to use again. for instance, you can use the format re-index to sql etc.
or if you want to
i completely forgot about this feature!!! i read about it when i first
started using mnogosearch, but never bothered to use it.
with mirror feature, wouldn't it be easy to implement Google's "cache"
feature where the user can view a cache of the page from the last time
you indexed.
I
Alexander Barkov wrote:
i completely forgot about this feature!!! i read about it when i first
started using mnogosearch, but never bothered to use it.
with mirror feature, wouldn't it be easy to implement Google's "cache"
feature where the user can view a cache of the page from the
--- Alexander Barkov [EMAIL PROTECTED] wrote:
Alexander Barkov wrote:
i completely forgot about this feature!!! i read about it when i
first
started using mnogosearch, but never bothered to use it.
with mirror feature, wouldn't it be easy to implement Google's
"cache"
Caffeinate The World wrote:
The only one disadvantage is that it will not work on huge
search engines with millions documents. There is a limit on total
file number on file system in most unixes.
For example, my 30G /usr partition on FreeBSD box can create about 8
mln
files.
is
--- Alexander Barkov [EMAIL PROTECTED] wrote:
Caffeinate The World wrote:
The only one disadvantage is that it will not work on huge
search engines with millions documents. There is a limit on total
file number on file system in most unixes.
For example, my 30G /usr partition on
The only one disadvantage is that it will not work on huge
search engines with millions documents. There is a limit on total
file number on file system in most unixes.
For example, my 30G /usr partition on FreeBSD box can create about 8
mln
files.
is that a per file system limit
Zenon Panoussis skrev:
By now, I have almost 1 GB of indexed files, 4 indexer
crashes and one splitter crash. I'll do the debugging and
post its output tomorrow.
===
# gdb indexer core.indexer.01
GNU gdb 5.0
Copyright 2000 Free Software Foundation, Inc.
GDB is free
I just have to put in my encounters here, because they seem very similar. I get a
large amount of information indexed, but upon trying to run splitter, it will core
dump somewhere midway through, and on one round left wierd directories in the $VAR/raw
directory:
[root@spider raw]# ls -al
Zenon Panoussis skrev:
Now for 31 MB adventures :)
# ./run-splitter -k
Sending -HUP signal to cachelogd...
Done
# ./run-splitter -p
Preparing logs...
Open dir '/var/mnogo3110/raw'
Preparing word log 982024900 [ 42176 bytes]
Preparing word log 982027284 [31465324 bytes]
i've been going through this and back again time and time again. what
would really be nice is indexer save the logs in a format that's easy
to use again. for instance, you can use the format re-index to sql etc.
or if you want to reindex again, you don't have to crawl through all
the external
Hi!
Can you guys give us a log file produced by splitter -p which caused
crash? We can't reproduce crash :-(
Caffeinate The World wrote:
i reported this problems a while back. i believe it's being worked on.
atleast the recently found the bug why it wasn't splitting out to FFF.
the seg
Alexander Barkov skrev:
Can you guys give us a log file produced by splitter -p which caused
crash? We can't reproduce crash :-(
Huh? splitter doesn't accept the -v5 argument, so it won't give
more detailed logs than the normal ones. The only log I had, that
to stdout, is the one I
in my tests your 3 little files wouldn't make a difference. he would
have to run splitter -p and splitter on all the files starting from the
first original RAW file, including all the 31 MB file. i believe in my
case it was the original 31mb file which caused the problem.
while processing the
Could you please put zipped /var/mnogo319/tree/12/B/12BFD000 and
a file /splitter/XXX.wrd with correspondent XXX.del which produce
crash somewhere on the net?
Zenon Panoussis wrote:
Alexander Barkov skrev:
Can you guys give us a log file produced by splitter -p which caused
crash?
Zenon Panoussis wrote:
Alexander Barkov skrev:
Could you please put zipped /var/mnogo319/tree/12/B/12BFD000 and
a file /splitter/XXX.wrd with correspondent XXX.del which produce
crash somewhere on the net?
http://search.freewinds.cx/logs/logs.tar.gz
Not Found
The requested URL
Zenon Panoussis skrev:
And a really HARD hang at the same place as before. So hard
that I can't even kill splitter.
BTW, although I couldn't kill splitter, I did find a core dump
in sbin. Here's the backtrace:
# gdb splitter core
GNU gdb 5.0
snip copyright
This GDB was configured
Zenon Panoussis skrev:
I'll delete the entire tree directory and start re-indexing from
scratch. I'll make and split a small file first, ca 5 MB, then a
31 MB file, if that works yet another 31 MB file, and so on until
I get in problems again. Will report back later this evening.
First
Author: Zenon Panoussis
Email: [EMAIL PROTECTED]
Message:
RH Linux 7.0, search 3.1.9, MySQL 3.23.29, cache mode, with the
new patches for cache.c and sql.c.
It happens all the time. It started happening when "maximum size"
31 MB log files were indexed, but by now it happens on any indexing,
i reported this problems a while back. i believe it's being worked on.
atleast the recently found the bug why it wasn't splitting out to FFF.
the seg fault happens during the splitter process and not index. i've
been splitter when the logs are at about 2 MB and i've not had
splitter core dump on
32 matches
Mail list logo
