On 11-Mar-2004 Ian Clarke wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Toad wrote: >| Unfortunately crawling freenet via HTTP will have the main effect of >| DoSing your freenet node, because every web download takes up a thread, >| and we therefore limit parallel HTTP downloads to 24-36. Ideally you'd >| want a real FCP spider; there must be one out there somewhere. > > You can download something which claims to do this from: > > http://127.0.0.1:8888/[EMAIL PROTECTED]/spider/5// > > I tried it (in a sandbox Linux account, which is absoltely the minimum > precaution anyone should take if running code downloaded from an > untrusted anonymous source) and it seems to work pretty nicely.
Yes, this is the same spider I use to generate DFI. Actually, I'm using an earlier version that I've hacked on quite a bit. :-) -- Conrad Sabatier <[EMAIL PROTECTED]> - "In Unix veritas" _______________________________________________ Support mailing list [EMAIL PROTECTED] http://news.gmane.org/gmane.network.freenet.support Unsubscribe at http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/support Or mailto:[EMAIL PROTECTED]