Thanks Yeah, "live http headers" is even a better choice. I couldn't remember its name.
As to the 304 not modified, yes, that's telling you the file hasn't been modified between the time you're reloading, but be careful -- make sure you have "live http headers" started and recording before you make your first http request. Your first request will return a 200 or 301, then the next time you're reloading that page -- it's going to give you a 304 -- because it's essentially sending the same request with the same serial number and the web server knows that. Stephan http://quickspikes.com On Apr 7, 3:22 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > Wow! > > Nice reply man! > > I saw the other post talking about robots.txt but my app is already > blessed and its not working. I'm using "live http headers" (a firefox > extension) and my app robots.txt is returning a 304 not modified. > > I'm already using google webmaster tools but now its useless because > googlebot is blocked. > > Thank's for your reply! > > #sorry for my english, i'm spanish > > On 7 abr, 02:34, stephan <[EMAIL PROTECTED]> wrote: > > > Masylum, > > According to Orion Henry last February 29th on this Google Group, he > > said "That's a feature that will be available for blessed accounts a > > little ways down the line." > > > On a side-note, when you make your request for "blessing" to them -- > > do let them know their own robots.txt is working -- but only by > > chance. > > If you go tohttp://heroku.com/robots.txtortohttp://www.heroku.com/robots.txt > > > You'll find that they're robots.txt is written correctly, it's just > > that it is returning the wrong http status code in both cases. > > It's returning a 301 instead of a 200 OK. A 301 status code means that > > the content of that file is being dynamically generated. > > > And the google bot won't recognize a robots.txt (or a sitemap) that > > returns a 301. Now luckily for them, and they probably knew that > > already since they're saying it will only "be available down the > > line", they're not blocking anything using that file. So the fact that > > google is not recognizing that file as valid doesn't matter. If the > > google bot doesn't see a valid robots.xml file with a right status > > code, its default behavior is to index everything anyway. > > > On the other hand, if you go to your own public app's robots.txt, > > let's say your app is called "foobar2000" > > You'll find thathttp://foobar2000.heroku.com/robots.txtretunsthe > > right 200 OK > > > That's because your own robots.txt request is being intercepted by > > their http/proxy web server, and the http web server will return the > > right 200 OK code. Ruby can also return the right 200 OK status code, > > it's just that you have to tell it explicitly to do so, and it's not > > an error most developers have come across unless they get first bitten > > by it. > > > To double-check the status codes of an http request, you use your own > > sniffer. An easy sniffer to install is "TamperData", a firefox > > extention.https://addons.mozilla.org/en-US/firefox/addon/966(itmust be > > enabled > > once installed, and then it must be explicitly started from its dialog > > menu) > > > Anyway, good luck Masylum, and if you haven't done so already -- try > > the "Google Webmaster Tools" when you get this robots.txt > > working.https://www.google.com/webmasters/tools/siteoverview?hl=en(it's an > > important tool) > > > - Stephanhttp://quickspikes.com > > > On Apr 6, 5:05 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > > > > Anybody knows how to change the robots.txt? > > > > I modified the file in the public folder but the bots remain blocked. > > > > Maybe I'm doing something wrong, but I need to be accesible for > > > googlebot at least. > > > > Thank's in advance! :) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/heroku?hl=en -~----------~----~----~----~------~----~------~--~---
