Hi, Peter Swoboda wrote: > Hi, > we're using Nutch 0.8. > In deafault.xml "ignore external links" is set "true". > Can anybody tell me where we can find the code to this property? > We've got the problem, that now, there are many "intern" pages, that > aren't indexed. > Doesn't seem to make sense, because they are on the same server, like > other indexed pages. > When we set "ignore external links" "false" they are indexed. > What could be the problem? > Do you have different hosts in your server?
ignore.external.links property, if set to true, ignores links whose _host_ is different from the source page. For example, Assume page www.bar.com/index.html contains a link to foo.bar.com/page.html. if ignore.external.links is true, host of the source page (www.bar.com) and host of the link (foo.bar.com) will be compared and since they are different this link will be ignored. Even though, they are probably on the same server. So only links within the exact same host (in this case, www.bar.com) are followed. -- Doğacan Güney > Peter > > > > . >
