Georg Brandl <g.brandl <at> gmx.net> writes:
>
> Guido van Rossum schrieb:
> > I think the better syntax would be to add site:mail.python.org to the
> > query, but you're right, that doesn't seem to find recent messages.
> > Maybe the absence of a robots.txt file on mail.python.org could be a
> > partial explanation?
>
> Doesn't the absence of a robots.txt mean "you may index everything"?
It does.
However, pages such as:
http://mail.python.org/pipermail/python-dev/
(and, it seems, all other pipermail-generated archive pages)
have the following HTML tag in them:
<META NAME="robots" CONTENT="noindex,follow">
which explicitly instructs Web spiders *not* to index contents nor follow links.
Regards
Antoine.
_______________________________________________
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
