Hi Phil, Thanks for the suggestion. Some of them are really nice. I never think of it.
Yes, the scraper follow links. Regards Leigh On 3 July 2014 15:45, Phil Scadden <[email protected]> wrote: > On 3/07/2014 2:01 p.m., Leigh Wanstead wrote: > > I have a question for you guys related to how to be a good developer > > to solve the problem :-) > Well I have done website scrapping - no browser involved - but with the > site owners blessing. Tough problem. Does the scraper follow links in > your site? You could put in a link that is hidden by css and return > garbage to anything that follows it. Make a very close study of header > etc of anything that might give them away. Dont use it block - use it to > generate garbage content for them instead. > > You could use js to get and display text with lazy loading. Or put text > in image/flash. That makes it a pain in the neck for scrapers. Vary the > dom layout at random but use matching css to make the appearance the > same for users - this doesnt stop scraping but makes finding the > relevant information a pain because to the scraper, the website is > constantly changing format. > > > Notice: This email and any attachments are confidential. > If received in error please destroy and immediately notify us. > Do not copy or disclose the contents. > > _______________________________________________ > NZ Borland Developers Group - Delphi mailing list > Post: [email protected] > Admin: http://delphi.org.nz/mailman/listinfo/delphi > Unsubscribe: send an email to [email protected] with > Subject: unsubscribe >
_______________________________________________ NZ Borland Developers Group - Delphi mailing list Post: [email protected] Admin: http://delphi.org.nz/mailman/listinfo/delphi Unsubscribe: send an email to [email protected] with Subject: unsubscribe
