Caffeinate The World skrev:
> 

> in my tests your 3 little files wouldn't make a difference. he would
> have to run splitter -p and splitter on all the files starting from the
> first original RAW file, including all the 31 MB file. i believe in my
> case it was the original 31mb file which caused the problem.

OK, I'll try to do this systematically now and write down everything 
I do. Bear with me if you get too much information at times; it's 
hard to know what could could be relevant and what not.

1. Installed v 3.1.10 with a new prefix (produced one error, see 
   the webboard on subject "warning: no newline at end of file")
2. Copied 319/var/* to 3110/var/ (oofff! took more than an hour)

   The raw directory looks like this: 
   -rw-r--r--    1 root     root        17096 Feb  7 07:58 981529092.del.done
   -rw-r--r--    1 root     root     10060900 Feb  7 07:58 981529092.wrd.done
   -rw-r--r--    1 root     root        16808 Feb  7 09:24 981534260.del.done
   -rw-r--r--    1 root     root     11374124 Feb  7 09:24 981534260.wrd.done
   -rw-r--r--    1 root     root        13400 Feb  7 11:03 981540190.del.done
   -rw-r--r--    1 root     root     11698476 Feb  7 11:03 981540190.wrd.done
   -rw-r--r--    1 root     root        20328 Feb  7 12:54 981546899.del.done
   -rw-r--r--    1 root     root      8055532 Feb  7 12:54 981546899.wrd.done
   -rw-r--r--    1 root     root         7312 Feb  7 14:52 981553965.del.done
   -rw-r--r--    1 root     root      4459360 Feb  7 14:52 981553965.wrd.done
   -rw-r--r--    1 root     root         9912 Feb  7 16:52 981561131.del.done
   -rw-r--r--    1 root     root      5254828 Feb  7 16:52 981561131.wrd.done
   -rw-r--r--    1 root     root        14240 Feb  7 18:53 981568430.del.done
   -rw-r--r--    1 root     root     10220088 Feb  7 18:53 981568430.wrd.done
   -rw-r--r--    1 root     root          216 Feb  7 22:27 981581773.del.done
   -rw-r--r--    1 root     root       220988 Feb  7 22:27 981581773.wrd.done
   -rw-r--r--    1 root     root        14088 Feb  8 22:40 981669855.del.done
   -rw-r--r--    1 root     root      8719924 Feb  8 22:40 981669855.wrd.done
   -rw-r--r--    1 root     root          136 Feb  8 23:05 981669947.del.done
   -rw-r--r--    1 root     root       125028 Feb  8 23:05 981669947.wrd.done
   -rw-r--r--    1 root     root         5288 Feb  9 01:51 981679960.del.done
   -rw-r--r--    1 root     root       396972 Feb  9 01:51 981679960.wrd.done
   -rw-r--r--    1 root     root         1192 Feb  9 03:32 981686015.del.done
   -rw-r--r--    1 root     root       693916 Feb  9 03:32 981686015.wrd.done
   -rw-r--r--    1 root     root         4008 Feb 11 21:56 981925017.del.done
   -rw-r--r--    1 root     root      1876884 Feb 11 21:56 981925017.wrd.done
   -rw-r--r--    1 root     root         4192 Feb 11 22:51 981928286.del.done
   -rw-r--r--    1 root     root      3349232 Feb 11 22:51 981928286.wrd.done
   -rw-r--r--    1 root     root         4096 Feb 11 23:45 981931533.del.done
   -rw-r--r--    1 root     root      1265304 Feb 11 23:45 981931533.wrd.done
   -rw-r--r--    1 root     root        12944 Feb 12 02:56 981945565.del
   -rw-r--r--    1 root     root      6801160 Feb 12 02:56 981945565.wrd
   -rw-r--r--    1 root     root         9024 Feb 12 04:10 981993028.del
   -rw-r--r--    1 root     root      3751064 Feb 12 04:10 981993028.wrd
   -rw-r--r--    1 root     root            0 Feb 12 16:50 del.log
   -rw-r--r--    1 root     root            0 Feb 12 16:50 wrd.log

*  As you see, no 31 MB files; last time I got them they produced 
   segfaults, so I deleted them and went on, leaving the pages they 
   contained for the next re-indexing. This means that words that 
   should be in the word files according to the mysql database are 
   not there. I don't think it matters, but I cannot be sure. Actually, 
   depending on how words are indexed, this could be the cause of 
   the current segfaults. However, even so, this wouldn't change 
   the fact that the 31 MB files caused segfaults in the first place, 
   before I deleted them.

   In this context I should also mention that I am using non-ECC 
   memory. If splitting depends on the integrity of the pre-existing 
   word files, an error that has been entered by bad copying/writing 
   would affect all subsequent splitting attempts.

   This is what the database looks like:

   #du -c -h tree
   1.6G    total

   #./indexer -S
             Database statistics

    Status    Expired      Total
   -----------------------------
         0       4240       4240 Not indexed yet
       200          0      38945 OK
       301          0         52 Moved Permanently
       302          0        312 Moved Temporarily
       304          0         65 Not Modified
       400          0          2 Bad Request
       403          0         35 Forbidden
       404          0       2133 Not found
       503          5          5 Service Unavailable
       504          1          1 Gateway Timeout
   -----------------------------
     Total       4246      45790

3. Started splitting with 3110 where I had left off in 319.

   # ./splitter -p
   Open dir '/var/mnogo3110/raw'
   Preparing word log 981945565
   Preparing word log 981993028
   Preparing del log 981945565
   Preparing del log 981993028

   #./splitter
   <snip long OK log>
   Delete from cache-file /var/mnogo3110/tree/12/B/12BF1000
   Delete from cache-file /var/mnogo3110/tree/12/B/12BF6000
   Delete from cache-file /var/mnogo3110/tree/12/B/12BFB000
   Delete from cache-file /var/mnogo3110/tree/12/B/12BFD000
   /var/mnogo3110/tree/12/C/12C03000 old:   1 new:   1 total:   2
   /var/mnogo3110/tree/12/C/12C0B000 old:   1 new:   1 total:   2
   /var/mnogo3110/tree/12/C/12C10000 old:  70 new:   3 total:  73
   /var/mnogo3110/tree/12/C/12C11000 old:   4 new:   1 total:   5
   /var/mnogo3110/tree/12/C/12C22000 old:   2 new:   1 total:   3
   /var/mnogo3110/tree/12/C/12C28000 old:  90 new:  11 total: 101
   /var/mnogo3110/tree/12/C/12C34000 old:  11 new:   4 total:  15
   /var/mnogo3110/tree/12/C/12C37000 old: 195 new:  15 total: 210
   /var/mnogo3110/tree/12/C/12C38000 old:  11 new:   2 total:  13
   /var/mnogo3110/tree/12/C/12C41000 old:  19 new:   1 total:  20
   /var/mnogo3110/tree/12/C/12C42000 old:  19 new:   1 total:  20

   And a really HARD hang at the same place as before. So hard 
   that I can't even kill splitter.

I'll delete the entire tree directory and start re-indexing from 
scratch. I'll make and split a small file first, ca 5 MB, then a 
31 MB file, if that works yet another 31 MB file, and so on until 
I get in problems again. Will report back later this evening. 

Z


-- 
oracle@everywhere: The ephemeral source of the eternal truth...
______________
If you want to unsubscribe send "unsubscribe udmsearch"
to [EMAIL PROTECTED]

Reply via email to