Hello Stefan,

 Big thanks!
 But also is interested for me: how you connect your database to
 searcher?
 I have changed "searcher.dir" to the directory where my db and
 segments. Searcher works fine, but does not consider these updates
 until I not restart Tomcat. ;-/
 It is bad for me...
 

Wednesday, November 3, 2004, 1:04:47 AM, you wrote:

>>
>> Can you show it please?
>>
SG> # as many you wish to have
SG> LIMIT=100
SG> for ((a=1; a <= LIMIT ; a++))
SG> do
SG> echo '************** start new crawl loop '$a'**************'
SG> #bin/nutch generate db segments -topN 1000000
SG> bin/nutch generate db segments -topN 50000 > gen_$a.log 2>&1
SG> cat gen_$a.log | mail -s'nutch generated segments '$a [EMAIL PROTECTED]
SG> cat gen_$a.log
SG> s1=`ls -d segments/2* | tail -1`
SG> echo $s1
SG> bin/nutch fetch $s1
SG> bin/nutch updatedb db $s1 > update_$a.log 2>&1
SG> cat update_$a.log  | mail -s'nutch update db '$a [EMAIL PROTECTED]
SG> cat update_$a.log
SG> bin/nutch analyze db 3
SG> # mail -s'analyzing db ready' [EMAIL PROTECTED]
SG> bin/nutch index $s1
SG> # mail -s'ready with indexing' [EMAIL PROTECTED]
SG> bin/nutch dedup segments dedup$a.tmp
SG> #echo $out | mail -s'ready with crawling' [EMAIL PROTECTED]
SG> du -hs db segments | mail -s'nutch db size after loop: '$a [EMAIL PROTECTED]
SG> done
SG> exit 0


-- 
Best regards,
 NGS                            mailto:[EMAIL PROTECTED]



-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
Nutch-developers mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/nutch-developers

Reply via email to