I'm running lighttpd, flup, and django 0.96. I recently switched to django for my web site. When I first did this, the search bots came and indexed it dutifully.
I recently wrote a bunch of copy for my site, and waited, pathetically tailing my access logs, watching for the next crawl. When googlebot finally visited, it made a couple of HEAD requests and left. Needless to say I was disappointed. So I tried to be the bot and looked at what it would have seen: $ curl -s -I http://****.com/HTTP/1.1 200 OK Vary: Cookie Content-Type: text/html; charset=utf-8 Date: Thu, 20 Sep 2007 15:44:33 GMT Server: lighttpd/1.4.13 I can see how conservative crawlers would decide to leave. What can I do to encourage the crawlers to actually crawl? -- Darrin --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
