On 5/15/24 19:15, Dave Wreski wrote:
Hi,
Google insists that one of our staging sites needs to be indexed despite
"disallow" in robots.txt and a half-dozen other methods for preventing
Google from indexing it (including submitting it for removal from their
index). The staging site is even protected with a RequireAll statement
for the DocumentRoot based on the IP, which then results in a 404 and
other errors in GSC. This impacts our SEO and also causes GSC to stop
processing the rest of our site.
The next steps I'd like to do is to redirect anyone not in that
RequireAll statement to be redirected to the production site. Is this
possible? Perhaps a RewriteCond that depends upon certain IPs, then
otherwise redirects to the production site?
Thanks,
Dave
The last time I ran into this was back in iptables days 20 years ago.
Based on IP they were denied because my site at the time included my
photo's and totalled about 13 gigabytes. This was in the days of
bandwidth per month of 30 gigs. Because google has so many machines they
used up all my allocation long before the month was up. I wound up
putting another search engine in that database, mj12, so I wound up with
an iptables file about 15k lines long. That continued until I had ported
the whole thing to a couple new Seacrate 1t drives, both of which went
tits down in the night within 2 weeks, just disappearing off the
sata-III bus. I was so pi$$ed I didn't even warranty them. SSD's are it
today. I have only one spinning rust drive in 8 machines here now, a 250
gig that refuses to die. iptables worked but took about 10 hours a month
to maintain it cuz they moved the machines to a new address. Some of
the iptables rules ended in /16, so I was blocking a goodly share of the
ipv4 space when I had the gran crash. I controlled it most of the time
but it was several hours a week keeping even with them. You never get
ahead. I still have a registered name but all you get is the apache test
page.
Cheers, Gene Heskett, CET.
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
- Louis D. Brandeis
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org
