On Tue, Jan 19, 2010 at 9:34 AM, Barbie <[email protected]> wrote: >> Does redirecting the MSN bot to the static site help at all? (I >> though the issue was that spidering the site triggers page >> regeneration.) > > They are able to index the static site already, as that's at > static.cpantesters.org, a completely different domain. And having just > checked indeed they are indexing it, albeit at a considerably slower > speed than the www.cpantesters.org domain.
I think I mixed up my terms. Could we turn www.cpantesters.org into a proxy for static.cpantesters.org -- but just for MSN bots. E.g. grab content from static and serve it back. David
