Hi, I our case in every crawl we get 80 to 90 % fetched urls as an average. If we got more than 85% we are assuming this as a successful crawl.
regards Neelesh Rathore ianwong wrote: > > hi, Neelesh > > What is your final ratio? did all urls are fetched? My crawler has > finished > it job. I just got the ratio from final log. > > Thanks > Ian > > -------------------------------------------------- > From: "Neelesh Rathore" <[EMAIL PROTECTED]> > Sent: Friday, December 05, 2008 4:17 PM > To: <[email protected]> > Subject: Re: nutch crawl - strange results > >> >> Hi Friend, >> As you know that I was facing the same situation, But the statistics >> ratio of fetched and un-fetched goes normal at the end of the crawl. I >> had >> observed that at the start of the crawl fetched url are very less as >> compared to total urls, but as crawl get end the ratio is fine. >> So please wait till your crawl get finished. >> >> >> >> >> ianwong wrote: >>> >>> I has similar ratio, any idea to decrease unfecthed urls? >>> >>> Ian >>> >>> >>> Neelesh Rathore wrote: >>>> >>>> hello every one, >>>> >>>> I had trying to do crawling on one url , and my crawling is still in >>>> progress , but when i had tryed to see the statstics by crawl readdb i >>>> got that the ratio of db-fetched and db-unfeched is very large. >>>> i found total urls around 20000 but the fatched urls are only 1600 . >>>> >>>> please suggest me what to do. >>>> >>>> Thanks >>>> >>>> Neelesh >>>> >>>> >>>> >>>> >>> >>> >> >> -- >> View this message in context: >> http://www.nabble.com/nutch-crawl---strange-results-tp9433066p20857332.html >> Sent from the Nutch - User mailing list archive at Nabble.com. >> >> > > -- View this message in context: http://www.nabble.com/nutch-crawl---strange-results-tp9433066p20889519.html Sent from the Nutch - User mailing list archive at Nabble.com.
