Unfortunately this is not something we can fix right now.
Best,
Josh
On Feb 17, 8:02 am, GA wrote:
> But you give away a lot of money if you block the AdSense crawler,
> because AdSense cannot deliver relevant ads, therefore the click rate
> is low and also the profit.
>
> On 17 Feb., 02:02, Josh (Google Employee) wrote:
>
> > On Feb 16, 1:51 pm, GA wrote:
>
> > > if you permit u=http in your robots.txt file, the AdSense crawler is
> > > allowed to crawl and the error message should
> > > disappear.http://translate.googleusercontent.com/robots.txt
>
> > We correctly block robots.txt from crawling our translations of
> > webpages. We don't want web crawlers to try to crawl complete
> > translations of the web through our service ;-)
>
> > Cheers,
> > Josh
