[ http://issues.apache.org/jira/browse/NUTCH-13?page=comments#action_63314 ] Matthias Jaekle commented on NUTCH-13: --------------------------------------
Adding localhost to the urlfilter does not solve the problem with the example url. And IPs are not compared by the the urlfilter. > If dns points to 127.0.0.1, the url is also crawled > --------------------------------------------------- > > Key: NUTCH-13 > URL: http://issues.apache.org/jira/browse/NUTCH-13 > Project: Nutch > Type: Bug > Components: fetcher > Reporter: Matthias Jaekle > Priority: Minor > > For example www.tik24.de points to 127.0.0.1. > If you follow a link to www.tik24.de fetcher will crawl content from your own > machine. > Wrong DNS entries could create unwanted entries in segments. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira ------------------------------------------------------- This SF.Net email is sponsored by: New Crystal Reports XI. Version 11 adds new functionality designed to reduce time involved in creating, integrating, and deploying reporting solutions. Free runtime info, new features, or free trial, at: http://www.businessobjects.com/devxi/728 _______________________________________________ Nutch-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-developers
