On Mon, Oct 3, 2011 at 8:55 AM, Ed Summers <e...@pobox.com> wrote: > Respecting robots.txt is key. It's normally easy to eyeball, but when > in doubt something like robotparser [1] can be helpful. A bit of > reflection on the Golden Rule probably is probably more important than > pondering the legality of what you are doing.
s/probably is/is/ //Ed