Hello, I am using nutch 1.4 to crawl a url . After crawling the content of segment is : Status: 1 (db_unfetched) Fetch time: Fri Jul 12 13:43:43 IST 2013 Modified time: Thu Jan 01 05:30:00 IST 1970 Retries since fetch: 0 Retry interval: 2592000 seconds (30 days) Score: 1.0 Signature: null Metadata: _ngt_: 1373616835706
Content:: Version: -1 url: http://farmer.gov.in/COLD_STROAGE_Link.aspx base: http://farmer.gov.in/COLD_STROAGE_Link.aspx contentType: text/html metadata: X-AspNet-Version=4.0.30319 Date=Fri, 12 Jul 2013 08:19:30 GMT Content-Length=170 nutch.crawl.score=1.0 Location=/(S(ulnaaubb1l0bku22vik2tzjt))/COLD_STROAGE_Link.aspx _fst_=35 nutch.segment.name=20130712134358 Content-Type=text/html; charset=utf-8 Connection=close Server=Microsoft-IIS/7.5 X-Powered-By=ASP.NETCache-Control=private Content: <html><head><title>Object moved</title></head><body> <h2>Object moved to <a href="/(S(ulnaaubb1l0bku22vik2tzjt))/COLD_STROAGE_Link.aspx">here</a>.</h2> </body></html> CrawlDatum:: Version: 7 Status: 35 (fetch_redir_temp) Fetch time: Fri Jul 12 13:44:03 IST 2013 Modified time: Thu Jan 01 05:30:00 IST 1970 Retries since fetch: 0 Retry interval: 2592000 seconds (30 days) Score: 1.0 Signature: null Metadata: _ngt_: 1373616835706_pst_: temp_moved(13), lastModified=0: http://farmer.gov.in/(S(ulnaaubb1l0bku22vik2tzjt))/COLD_STROAGE_Link.aspx Thing is my seed url is redirected to a diiferent url . But problem is the content of this redirected url is not fetched by nutch. I have changed rediect.max to 5 . But still content is not fetched .Please help
