On Thu, 3 Oct 2019 at 12:52, Vladimir Sitnikov <[email protected]> wrote: > > sebb>So surely that means it won't return matches to the staging site based > on content? > > Which documentation do you read?
As below: "Google won't crawl or index the content blocked by robots.txt" If Google does not index the content, it can't return searches for text in the content. > As for me, Google WILL display those results. If they don't have the content, how can they return matching results? > - While Google won't crawl or index the content blocked by robots.txt, > we might still find and index a disallowed URL if it is linked from other > places on the web. As a result, the URL address and, potentially, *other > publicly available information such as anchor text in links to the page can > still appear in Google search results*. To properly prevent your URL > from appearing in Google Search results, you should password-protect the > files on your server <https://support.google.com/webmasters/answer/93708> > or use the noindex meta tag or response header > <https://support.google.com/webmasters/answer/93710> (or remove the page > entirely). > > > They recommend password-protect the page though. > > So the only viable options so far are: > 1) drop the page > 2) password-protect it > > Vladimir
