I am currently rying to do my own version of a web crawler it gets the list
of sites to crawl from a separate text file and attenmps to visit them but
after a few sites I get th following error and it quits to the console and
even there afterwards I can open up any other web site:

cannot find it
connecting to: www.multimania.com
connecting to: www.multimania.com
** Access Error: Cannot connect to www.multimania.com.
** Where: as: read join http:// [url]
>>

does rebol have a buffer?? here is my code

REBOL [
    Title:   "e-mail finder"
    Date:    13-May-2000
    Author:  "Stephane Jolicoeur"
    File:    %octo.r
    Email:   [EMAIL PROTECTED]
    Purpose: {
        To find urls within a file!!!
    }
    Comments: {
        do not use for SPAM
    }
]
urls: make block!
text: make string! 0
html-code: [
 thru "http://" copy url to newline (append urls url) |
 copy txt to "http:" (append text txt)
]
page: read %urls.txt
parse page [to "http://" some html-code]
foreach url urls

 if exists? join http:// [url] [
  as: read join http:// [url]
   ;print url
   if find as "@" [
    print ["@" "fut trouver sur" url]
    newline
    print " "
   ] print "cannot find it"
  clear as
  ][print ["je ne peux acceder ce site:" url]]
];
as: ask "done??"

thanks for any help
Akhar


Reply via email to