try this
in crawl-urlfilter.txt
+^http://([a-z0-9]*\.)*
On 1/14/07, visava <[EMAIL PROTECTED]> wrote:
Hi
I have created a file under urls directory with list of urls.
I have changed the crawl-urlfilter.txt to so that all urls in my list are
crawled.but it is not crawling anything
+^http://([a-z0-9]*\.)*.com/
only when I change the expression to something like
+^http://([a-z0-9]*\.)*yahoo.com/ will it crawl that particular domain.
Is the intranet crawler meant only for crawling one particular domain.
my command is
bin/nutch crawl urls -dir crawl.test -depth 3
Thanks
Harish
--
View this message in context:
http://www.nabble.com/crawling-url-list-tf2983090.html#a8330415
Sent from the Nutch - User mailing list archive at Nabble.com.
--
www.babatu.com
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
