why i can crawl http://game.search.com but i can't crawl http://www.search.com? conf/crawl-urlfilter is
# skip file:, ftp:, & mailto: urls -^(file|ftp|mailto): # skip image and other suffixes we can't yet parse #-\.(png|PNG|ico|ICO|css|sit|eps|wmf|zip|mpg|gz|rpm|tgz|mov|MOV|exe|bmp|BMP)$ # skip URLs containing certain characters as probable queries, etc. [EMAIL PROTECTED] # skip URLs with slash-delimited segment that repeats 3+ times, to break loops -.*(/.+?)/.*?\1/.*?\1/ # accept hosts in MY.DOMAIN.NAME #+^http://([a-z0-9]*\.)*search.com/ # skip everything else +. and some host i can't crawl because have error "Generator: 0 records selected for fetching, exiting ..." i set the same config for all host.why? -- View this message in context: http://www.nabble.com/Nutch-crawl-problem-tp14327978p14657080.html Sent from the Hadoop Users mailing list archive at Nabble.com.
