On Fri, 2003-10-31 at 17:11, tripolar wrote:
> Hello all
> I need a program to crawl links in freenet to get sites in cache before 
> I need them. I get frustrated with the speed of freenet and dead links. 
> I have used freenet on windows & linux and just installed freenet last 
> night on this winbox. I spent hours clicking on links just trying to 
> pull them in.
> Anything I can do other than manually clicking on all links?

A winbox, eh?

Well, there are dozens of freeware and shareware (crack available)
windoze programs to recursively download websites. Sites like
www.tucows.com, www.nonags.com, www.shareware.com etc list them by the

You might need to try a few until you hit on one which doesn't molest
the '//' in freenet URIs.

Once you choose a site downloader program, you'd need to point it at:
- http://localhost:8888/[EMAIL PROTECTED]/TFE//
- http://localhost:8888/[EMAIL PROTECTED]/YoYo//
or, point it at whatever site(s) you need, observing the above URL
syntax. Be sure to disable the timeout (or extend it to an hour or so),
because these crawler progs are used to web performance.

One last thing - these crawler progs won't be able to tell one freesite
from another, since it will perceive all freesites as part of the same
'site' at http://localhost:8888/. So take care to set a pattern match
requirement (unless you want the crawler to suck the whole freenet).


Support mailing list

Reply via email to