I don't know if any of you have been following the bruhaha about aggregators downloading whole websites, but I ran across the following message from Mark Pilgrim today, and wondered if Plucker wanted to start being a good web-citizen and honouring robots.txt.
http://diveintomark.org/archives/2003/02/21/newsmonster_day_2.html#c0004 03 Later, Blake. _______________________________________________ plucker-dev mailing list [EMAIL PROTECTED] http://lists.rubberchicken.org/mailman/listinfo/plucker-dev
