If you have that many files in a directory, I would try a filesystem that indexes directories. Reiserfs does, and there's a patch somewhere for ext2 (and probably for ext3). I don't know about the other filesystems.
Otherwise the application has to do a linear search through the directory every time it operates on a filename. As to why you're hitting a wall, perhaps you're blowing through a cache in MySQL or the kernel. Check number of allowed open files (ulimit -n), and whatever tweaks there are in the kernel (in /proc probably) for size of directory caches (can't help you there but I'm sure someone can). --Pete On Tue, Mar 04, 2003 at 09:04:20PM -0500, Gary Huntress wrote: > Greetings, > > I've been running a public MySQL server for a couple of years that now > supports several thousand users. I used to run this on a lowly PII-350 and > it ran quite fine with CPU loads usually hanging well under 0.20. The only > hitch was the time necessary to connect seemed to jump up to a few seconds > when the number of databases exceeded 2000 or so. It appeared to not be > linear, performance was "good" below 2000 databases and the connection time > went to a couple of seconds (as measured by a trivial php page that makes a > connection). > > I suspected that the connection speed was an ext2 filesystem bottleneck > when it had to deal with the large number of files in the mysql data > directory. > > I modernised a few months ago to an athlon 1.3GHz with the same ram and > a faster hard drive. I also upgraded to Linux 7.3 using an ext3 filesystem. > I thought that ext3 might handle a large number of files better, and I also > thought that the faster box and newer kernel would allow me to have more > databases on one server without suffering the same connection speed penalty. > However, I seem to have hit this same wall at roughly the same number of DBs > (closer to 3000 this time) > > To summarize, it appears to me that there is a reasonable upper bound on > the number of databases without suffering connection performance, probably > limited by the filesystem. > > My questions are 1) Are these observations expected? (I was a bit > disappointed that the new box didn't perform better...the cpu load is > usually idle) 2) Is there a way to mitigate this at the linux admin > level? (kernel parameters? ext3 parameters? change to reiserfs? 3) > Finally, are there MySQL parameters that can be tuned to enhance performance > for a large number of databases (note, not concurrent users...that is a > small number) > > > Regards, > Gary "SuperID" Huntress > ======================================================= > FreeSQL.org offering free database hosting to developers > Visit http://www.freesql.org > > > --------------------------------------------------------------------- > Before posting, please check: > http://www.mysql.com/manual.php (the manual) > http://lists.mysql.com/ (the list archive) > > To request this thread, e-mail <[EMAIL PROTECTED]> > To unsubscribe, e-mail <[EMAIL PROTECTED]> > Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php > --------------------------------------------------------------------- Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail <[EMAIL PROTECTED]> To unsubscribe, e-mail <[EMAIL PROTECTED]> Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
