On Sat, 2005-04-16 at 13:21 +0200, Boris Kroeger wrote: > One file of mine contained an exclamation mark (!) and was not processed > by the nutch crawl. > After I removed it nutch was able to process it. > May be there are further characters? > > Is this worth an issue in JIRA?
See crawl-urlfilter.txt and regex-urlfilter.txt: # skip URLs containing certain characters as probable queries, etc. [EMAIL PROTECTED] (another minor but important difference between filesystem indexing and web indexing)
