*********** REPLY SEPARATOR ***********
On 21/10/2008 at 3:37 p.m. Berend de Boer wrote:
>Who doesn't know that .nz sites pay heavily for international traffic?
>What crawler, unless you built one for the first time, doesn't
>understand that robots.txt is there for the protection of the crawler,
>so it doesn't get sucked in?
...and who would realistically expect a New Zealand organisation to use
an off-shore data trawler to skim NZ content?!
Am I the only one who sees this at moronic behaviour?!
On a side note: Can someone please remind me how to block certain IP
addresses from having ANY access? (Assuming best done in .htaccess) After more
than a week of broken sleep thanks to an insomniac baby, I can hardly remember
which hole in my head to put the toothbrush in each morning!
---
Karl
Senior Account Manager
www.KIWIreviews.co.nz ... Where Your Views Count
Please consider the environment before printing this email.
Supporting Palmerston North's Santa... see our Community Gold Project page:
http://www.KIWIreviews.co.nz/santa - To be seen on TVNZ's 'Mucking In' show!
--~--~---------~--~----~------------~-------~--~----~
NZ PHP Users Group: http://groups.google.com/group/nzphpug
To post, send email to [email protected]
To unsubscribe, send email to
[EMAIL PROTECTED]
-~----------~----~----~----~------~----~------~--~---
