Okay, I have ruled out bots. After disallowing all user-agents for a few days the load did not decrease, my site continued having errors. I have also, with the help of my host, ruled out DOS attacks. I have also optimized the database. I don't know what else to try, my users are still getting this error every couple of minutes.
Service Temporarily Unavailable The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later. Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request. It seems when analyzing the load factor compared to the time the errors appear that the load steadily increases to a factor of about 25, then the error appears, the load decreases, and it goes back online. My host said this was the way that the server behaves when under a heavy load of legit users and that I should focus on optimizing the database. However, this is a brand new quad-core server. My old server (dual core, half the ram) was running the same exact database with no problems. Does anyone have any ideas? On Mon, Feb 2, 2009 at 3:34 AM, Benjamin Lees <[email protected]> wrote: > Google offers such a tool with their Webmaster Tools < > https://www.google.com/webmasters/tools>; I'm sure there are plenty of > similar tools out there. > > On Sun, Feb 1, 2009 at 1:17 PM, Philip Beach <[email protected]> > wrote: > > > Ok thanks, but how can I be sure it's working before all my pages drop > > off google (if it were wrong). Is there some way to validate it in > > the context of my site? > > > > Thanks again > > > > > > On 2/1/09, Benjamin Lees <[email protected]> wrote: > > > TryUser-agent: * > > > Disallow: /index.php > > > Disallow: /skins/ > > > Disallow: /Special:Search > > > Disallow: /Special:Random > > > > > > Some other good rules to include are > > > Disallow: /MediaWiki: > > > Disallow: /Template: > > > > > > and maybe > > > Disallow: /Category: > > > > > > On Sun, Feb 1, 2009 at 2:06 AM, Philip Beach <[email protected]> > > wrote: > > > > > >> Haha true about rotting RAM, I'll look into that. I am not using > > wikimedia > > >> style URL's, sadly :( it just didn't happen when the site was first > set > > up > > >> and I can't move it now, for various reasons. All of my files are in > the > > >> web-root /. However, through an apache alias, my url is > > >> mywiki.com/Pagename. > > >> > > >> How would robots.txt look for that? Would I simply drop the preceeding > > >> /wiki, like this? > > >> > > >> User-agent: * > > >> Disallow: /Special:Search > > >> Disallow: /Special:Random > > >> > > >> Thanks a ton! > > >> > > >> On Sun, Feb 1, 2009 at 1:47 AM, Benjamin Lees <[email protected]> > > >> wrote: > > >> > > >> > Assuming you have Wikimedia-style URLs: > > >> > User-agent: * > > >> > Disallow: /w/ > > >> > Disallow: /wiki/Special:Search > > >> > Disallow: /wiki/Special:Random > > >> > > > >> > Your server will be able to handle a lot more if you set up as much > > >> caching > > >> > as you can <http://www.mediawiki.org/wiki/Manual:Cache>. No sense > > >> letting > > >> > all that spare RAM rot. :) > > >> > > > >> > > > >> > On Sat, Jan 31, 2009 at 10:02 PM, Philip Beach < > > [email protected] > > >> > >wrote: > > >> > > > >> > > I already have checked the access logs. It appears that Google and > > >> Yahoo > > >> > > are > > >> > > indeed generating a lot of traffic. Good idea Rob, I've been > working > > >> > > on > > >> > > this > > >> > > for a while. > > >> > > > > >> > > Just out of curiosity, what should my robots.txt look like for > > >> Mediawiki. > > >> > > Does anything need to be disallowed? > > >> > > > > >> > > On Sat, Jan 31, 2009 at 8:30 PM, Platonides <[email protected] > > > > >> > wrote: > > >> > > > > >> > > > You should check the access logs for which is causing the error. > > >> > > > > > >> > > > > > >> > > > _______________________________________________ > > >> > > > MediaWiki-l mailing list > > >> > > > [email protected] > > >> > > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l > > >> > > > > > >> > > _______________________________________________ > > >> > > MediaWiki-l mailing list > > >> > > [email protected] > > >> > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l > > >> > > > > >> > _______________________________________________ > > >> > MediaWiki-l mailing list > > >> > [email protected] > > >> > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l > > >> > > > >> _______________________________________________ > > >> MediaWiki-l mailing list > > >> [email protected] > > >> https://lists.wikimedia.org/mailman/listinfo/mediawiki-l > > >> > > > _______________________________________________ > > > MediaWiki-l mailing list > > > [email protected] > > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l > > > > > > > _______________________________________________ > > MediaWiki-l mailing list > > [email protected] > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l > > > _______________________________________________ > MediaWiki-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/mediawiki-l > _______________________________________________ MediaWiki-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-l
