Dean - I meant the IP of the crawler - we have lots of DENY ACLs in
place to curb rogue bots.

On Jun 11, 3:21 pm, "Dean Collins" <> wrote:
> Of course it is.
> Twitter were asked what "defines" a "bad" site on the second day but I
> haven't seen a reply apart from more questions about who is making the
> choice, eg will pornography be classed as "bad", will religious free
> speech be classed as "bad".
> I don't think the Twitheads thought through what it means to now offer
> an "aol" version of the web and the long term responsibilities that this
> entails through implicit guarantees to their users.
> Of course Ken you don't expect them to publish their ip address list do
> you....otherwise some smartass would route this ip address to a "clean"
> site and everyone else to the "bad" content.
> Regards,
> Dean Collins
> Cognation Inc
> <> +1-212-203-4357   New York
> +61-2-9016-5642   (Sydney in-dial).
> +44-20-3129-6001 (London in-dial).
> ________________________________
> From:
> [] On Behalf Of John
> Adams
> Sent: Friday, 11 June 2010 6:00 AM
> To:
> Subject: Re: [twitter-dev] tco crawler details
> is not a crawler; Are you referring to the URL unpacking process or
> something else?
> -john
> On Thu, Jun 10, 2010 at 11:46 PM, Ken <> wrote:
> If tco is to be the new three-letter agency and gatekeeper, we would
> like to treat it nice and whitelist its crawler. If tco is
> inadvertantly blocked, what happens?
> I do not know if we have already been checked by tco as I have not
> sent or received a dm with one of our own URLs.
> What are the user-agent and IP addresses used by this crawler? Does it
> check robots.txt?
> And since, for some, a tco thumbsdown could be a problem, is there a
> (speedy) appeals process?

Reply via email to