I am crawling an intranet. Apparently Nutch also indexes the url path
names (as a document) as it crawls. So if a query word appears in the
path name, the entire url path name would be one result. Since this
kind of info would typically be of no value to users, I want to filter
them out.
I think we have to crawl them since we need to get the actual document
urls underneath the path. But we do not want to index them. Is there
anyway to configure not to index path names during the crawling step?
If not, can we configure it in the search step? I know we can always
filter it using getDetails(). But this seems not a very clean way.
Thanks,
--Jay
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
