On Fri, 6 Aug 2004, [EMAIL PROTECTED] wrote: On Aug 6, 2004, at 6:14 AM, Ted Zlatanov wrote:
>> You misunderstand. If registration is required, a crawler will fail >> anyway, > > Unless the crawler is itself registered. If I wrote a crawler, I'd > keep a database of usernames and passwords for this purpose. That's not a typical web crawler, and obviously not what I meant. Such databases already exist (e.g. bugmenot) but using them to rip a page is definitely abusive. Think Google, not rip-off. Ted _______________________________________________ Boston-pm mailing list [EMAIL PROTECTED] http://mail.pm.org/mailman/listinfo/boston-pm
