Try this +^http://([a-z0-9\.\-]*)\.yahoo\.com/
I hope it help you :) -----Original Message----- From: zhangjin (JIRA) [mailto:[EMAIL PROTECTED] Sent: Monday, July 04, 2005 6:42 AM To: [email protected] Subject: [jira] Created: (NUTCH-67) I want crawl the websites including news.yahoo.com,game.yahoo.com,blog.yahoo.com,etc! I want crawl the websites including news.yahoo.com,game.yahoo.com,blog.yahoo.com,etc! ---------------------------------------------------------------------------- ----------- Key: NUTCH-67 URL: http://issues.apache.org/jira/browse/NUTCH-67 Project: Nutch Type: Wish Components: fetcher Environment: Windows 2000,weblogic Reporter: zhangjin how do I config them in the crawl-urlfilter.txt? I config them below,but it is not successful. # The url filter file used by the crawl command. # Better for intranet crawling. # Be sure to change MY.DOMAIN.NAME to your domain name. # Each non-comment, non-blank line contains a regular expression # prefixed by '+' or '-'. The first matching pattern in the file # determines whether a URL is included or ignored. If no pattern # matches, the URL is ignored. # skip file:, ftp:, & mailto: urls -^(file|ftp|mailto): # skip image and other suffixes we can't yet parse -\.(gif|GIF|jpg|JPG|ico|ICO|css|sit|eps|wmf|zip|ppt|mpg|xls|gz|rpm|tgz|mov|M OV|exe)$ # skip URLs containing certain characters as probable queries, etc. [EMAIL PROTECTED] # accept hosts in MY.DOMAIN.NAME #+^http://([a-z0-9]*\.)*MY.DOMAIN.NAME/ +^http://([a-z0-9]*\.)*yahoo.com/ # skip everything else #-. but It can not work, and can not crawl the domain name (DOMAIN.NAME) inluding news.yahoo.com,game.yahoo.com,blog.yahoo.com why? -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira ------------------------------------------------------- SF.Net email is sponsored by: Discover Easy Linux Migration Strategies from IBM. Find simple to follow Roadmaps, straightforward articles, informative Webcasts and more! Get everything you need to get up to speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click _______________________________________________ Nutch-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-developers
