* Dave Baker <dbkr at freenetproject.org> [2006-11-25 22:02:58]: > On Saturday 25 November 2006 21:48, sich wrote: > > toad a ?crit : > > > Why do you need it to survive the HTML filter? Is there some benefit to > > > fetching stuff through fproxy? > > > > I currently use mnogosearch as spider to find new freesite, and he use > > fproxy (I knon it's not the best way, but I am a poor devl). > > If you're not that confident with development, why not use one of the > plethora > of existing Freenet spiders rather than writing yet *another* one?
Well, when he started writing his spider, Librarian didn't exist ;) > > > If I can find this meta in the site this will be easier ton sort the > > sites on the index. > > I currently fetch : > > - Title > > - Content-Language > > - Description > > > > When I have find the sites with mnogosearch, I use some php scripts that > > I have write (they use FCP), to test if the site is responding, if we > > have the last version, etc. > > > > Toad is right here ... your spider ought to use FCP and not fproxy and its content-filter. -- NextGen$. "On peut ob??r aux lois en souhaitant qu'elles changent, comme on sert ? la guerre en souhaitant la paix." Merleau Ponty - L'?loge de la philosophie -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: <https://emu.freenetproject.org/pipermail/tech/attachments/20061126/25bba61c/attachment.pgp>
