buy the way is it fixed up yet(the wiki)?
On Fri, 10 Dec 2004 11:57:34 +0100 (CET), [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Hi, > > + more documentation is in the wiki, but a friendly scrip kiddy had deleted > our > wiki some weeks ago. We are rebuilding it actually. > > + How many crawling iterations you had done? Nutch works different then wget, > you may have to repeat the process several times. > > HTH > Stefan > > Zitiere uli <[EMAIL PROTECTED]>: > > > > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > hi there! > > > > my name is uli. > > > > in a project im searching for a good searchengine. i want to gather a > > lot of math-institutes, > > to create a math-specific search-engine. i tried some tools before and > > now i foud nutch. > > so now i got some questions: > > > > at the moment i am testing some examples. but in the beginning there > > is one big problem. > > different gatherers have different results and i dont now why!!! maybe > > u can help me and say me > > why the results are that different. > > trying wget --mirror www.mathematik-21.de gives 690 files (wget > > reports that, and i checked that, > > there are 690 docs and pics and so on) > > running nutch on that same domaein reports: > > > > 041208 165527 status: 310 pages, 226 errors, 1225553 bytes, 574098 ms > > ..... > > 041208 165532 done indexing > > 041208 165532 Reading url hashes... > > 041208 165532 Sorting url hashes... > > 041208 165532 Deleting url duplicates... > > 041208 165532 Deleted 0 url duplicates. > > 041208 165532 Reading content hashes... > > 041208 165532 Sorting content hashes... > > 041208 165532 Deleting content duplicates... > > 041208 165532 Deleted 3 content duplicates. > > 041208 165532 merging segment indexes to: www.mathe.de/index > > 041208 165532 merging segments _8t (310 docs) into _0 (307 docs) > > 041208 165532 done merging > > > > ok, nutch doesnt get gifs, jpegs and stuff, so i counted those > > formates, and i got still a difference. > > > > ~ 690 number of total docs in my wget mirror > > - - 132 number of gifs > > - - 8 number of tar.gz's > > - - 8 number of ppts > > - - 3 number of css > > - ------------------------------------------- > > ~ 539 docs left > > > > i checked for the whole list in conf/crawl-urlfilter.txt. i removed > > the line, that forbids =? and so on. > > > > i dont know what this difference could make. anyone of u have any > > ideas, what the reason could be. > > > > the next question is: where can i get a good documentation fo ur > > project. checked the tutorial, but thats > > really basic. id like to get more into details even in the sourcecode, > > and how lucene is embeded and so on. > > i found that there is something like CrawlTool (javadoc) and i think > > im gonna start reading some sourcecode, but > > maybe u have a documentation, that makes starting easier... > > > > so i send u regards, > > uli > > - -- > > > > > > > > \www.gnupg.org > > \find public key here: http://www.schinz.de/public.key > > > > -----BEGIN PGP SIGNATURE----- > > Version: GnuPG v1.2.6 (Darwin) > > Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org > > > > iD8DBQFBuX5lMqn9lZYnqAURAgddAKDBK86ouj9pJPzUq9DQFbbMdqLkeQCg5Q4o > > 5NdBMqtGBtuKZQQVDFDhFvA= > > =DuWb > > -----END PGP SIGNATURE----- > > > > > > > > ------------------------------------------------------- > > SF email is sponsored by - The IT Product Guide > > Read honest & candid reviews on hundreds of IT Products from real > > users. > > Discover which products truly live up to the hype. Start reading now. > > http://productguide.itmanagersjournal.com/ > > _______________________________________________ > > Nutch-developers mailing list > > [EMAIL PROTECTED] > > https://lists.sourceforge.net/lists/listinfo/nutch-developers > > > > > > ------------------------------------------------------- > SF email is sponsored by - The IT Product Guide > Read honest & candid reviews on hundreds of IT Products from real users. > Discover which products truly live up to the hype. Start reading now. > http://productguide.itmanagersjournal.com/ > _______________________________________________ > Nutch-developers mailing list > [EMAIL PROTECTED] > https://lists.sourceforge.net/lists/listinfo/nutch-developers > ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/ _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers
