Update: while I had to bounce Apache a number of times during the
afternoon/evening as a stopgap solution, I think I've managed to block
the bots from crawling the problem areas for now.
There will likely be more downtime in the future as I clean up the
existing solution a bit so it's a better long-term one (or if the
problem recurs.) However, with luck it will be minimal.
Thanks,
Jeff
On Fri, Nov 13, 2009 at 4:15 PM, Jeff Forcier j...@bitprophet.org wrote:
Hi all,
Just a note in case anyone has been trying to access the server, it
looks like some search engine bot is crawling the site and causing it
to overheat, so to speak (going OOM in a big way.) Not 100% positive
that's the cause, but _some_ traffic pattern in the past few hours is
causing the Redmine instance to gobble RAM where it was fine for the
last couple of months.
Thus, I've had to restart or shut down Apache a couple of times
recently, meaning *.fabfile.org has been intermittently down for the
count.
I'll be re-examining my Passenger config to see why it's OOMing
instead of simply refusing requests (probably need to rebalance
#-workers / allowed max memory / etc sorts of settings) in addition to
probably blacklisting this web crawler in my robots.txt. Hopefully
within the next hour or two, tops.
Thanks for your patience :)
-Jeff
--
Jeff Forcier
Unix sysadmin; Python/Ruby developer
http://bitprophet.org
--
Jeff Forcier
Unix sysadmin; Python/Ruby developer
http://bitprophet.org
___
Fab-user mailing list
Fab-user@nongnu.org
http://lists.nongnu.org/mailman/listinfo/fab-user
