Re: Question about crawler.

2010-04-20 Thread Phil Barnett
I meant the production 1.0 server is still crawling them.

Re: Question about crawler.

2010-04-20 Thread Phil Barnett
On Tue, Apr 20, 2010 at 7:02 PM, wrote: > Hi Phil, > > > -Original Message- > > From: Phil Barnett [mailto:ph...@philb.us] > > Sent: Wednesday, 21 April 2010 8:39 AM > > To: nutch-user@lucene.apache.org > > Subject: Question about crawler. > &g

RE: Question about crawler.

2010-04-20 Thread Arkadi.Kosmynin
Hi Phil, > -Original Message- > From: Phil Barnett [mailto:ph...@philb.us] > Sent: Wednesday, 21 April 2010 8:39 AM > To: nutch-user@lucene.apache.org > Subject: Question about crawler. > > Is there some place to tell why the crawler has rejected a page? I'm >

Question about crawler.

2010-04-20 Thread Phil Barnett
Is there some place to tell why the crawler has rejected a page? I'm trying to get 1.1 working and basically it doesn't seem to crawl the same way that 1.0 does. I have tika included in the parse- section of conf/nutch-site.xml I have DEBUG set for all the crawl sections, but it doesn't really sa