[ https://issues.apache.org/jira/browse/NUTCH-864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916840#action_12916840 ]
Doğacan Güney commented on NUTCH-864: ------------------------------------- I don't think that's possible to do without doing a DataStore#get first as we do not want to override current status on URL. I guess we could write redirect status as a temporary status somewhere, but it would be too complex IMHO. Julien, any ideas on how to set a redirect status without overwriting the current one? > Fetcher generates entries with status 0 > --------------------------------------- > > Key: NUTCH-864 > URL: https://issues.apache.org/jira/browse/NUTCH-864 > Project: Nutch > Issue Type: Bug > Components: fetcher > Environment: Gora with SQLBackend > URL: https://svn.apache.org/repos/asf/nutch/branches/nutchbase > Last Changed Rev: 980748 > Last Changed Date: 2010-07-30 14:19:52 +0200 (Fri, 30 Jul 2010) > Reporter: Julien Nioche > Assignee: Doğacan Güney > Fix For: 2.0 > > > After a round of fetching which got the following protocol status : > 10/07/30 15:11:39 INFO mapred.JobClient: ACCESS_DENIED=2 > 10/07/30 15:11:39 INFO mapred.JobClient: SUCCESS=1177 > 10/07/30 15:11:39 INFO mapred.JobClient: GONE=3 > 10/07/30 15:11:39 INFO mapred.JobClient: TEMP_MOVED=138 > 10/07/30 15:11:39 INFO mapred.JobClient: EXCEPTION=93 > 10/07/30 15:11:39 INFO mapred.JobClient: MOVED=521 > 10/07/30 15:11:39 INFO mapred.JobClient: NOTFOUND=62 > I ran : ./nutch org.apache.nutch.crawl.WebTableReader -stats > 10/07/30 15:12:37 INFO crawl.WebTableReader: Statistics for WebTable: > 10/07/30 15:12:37 INFO crawl.WebTableReader: TOTAL urls: 2690 > 10/07/30 15:12:37 INFO crawl.WebTableReader: retry 0: 2690 > 10/07/30 15:12:37 INFO crawl.WebTableReader: min score: 0.0 > 10/07/30 15:12:37 INFO crawl.WebTableReader: avg score: 0.7587361 > 10/07/30 15:12:37 INFO crawl.WebTableReader: max score: 1.0 > 10/07/30 15:12:37 INFO crawl.WebTableReader: status 0 (null): 649 > 10/07/30 15:12:37 INFO crawl.WebTableReader: status 2 (status_fetched): > 1177 (SUCCESS=1177) > 10/07/30 15:12:37 INFO crawl.WebTableReader: status 3 (status_gone): 112 > 10/07/30 15:12:37 INFO crawl.WebTableReader: status 34 (status_retry): > 93 (EXCEPTION=93) > 10/07/30 15:12:37 INFO crawl.WebTableReader: status 4 (status_redir_temp): > 138 (TEMP_MOVED=138) > 10/07/30 15:12:37 INFO crawl.WebTableReader: status 5 (status_redir_perm): > 521 (MOVED=521) > 10/07/30 15:12:37 INFO crawl.WebTableReader: WebTable statistics: done > There should not be any entries with status 0 (null) > I will investigate a bit more... -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.