Slow directory access with lots of files
Hello, I'm new to this list so please excuse me if this has been asked before or if I don't provide enough info. I'm having an issue and I hope someone can help me understand what is happening. I have a FreeBSD server that is running Samba. When users access directories that store lots of files (3k or so), access to the files is very slow. I'm guessing that it is a file system issue. I'd like to troubleshoot the problem but I don't know where to start. (In the mean time I have just told the users to create sub-directories and sort their files). Is there anything that I can do to make things run faster? Here is my OS version: # uname -a FreeBSD server.example.com 5.2.1-RELEASE FreeBSD 5.2.1-RELEASE #0: Mon Feb 23 20:45:55 GMT 2004 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC i386 Here is some of the software used by the users to access files (installed via ports): samba-3.0.4,1 nss_ldap-1.204_5 openldap-server-2.1.30 (plenty of others) I'm using POSIX ACL's on the files/directories that are slow. User account information is stored in LDAP. The server is a fast enough, uses RAID 5 w/ SCSI drives, and there is plenty of RAM. Utilization is fairly low so I don't think it is a HW problem. I'm going to update the OS and ports on this server soon. Perhaps that will help, but I'm not too hopeful. Any thoughts? Thanks, Totem ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Slow directory access with lots of files
On Mon, Jan 17, 2005 at 11:09:52AM -0800, Totem wrote: Hello, I'm new to this list so please excuse me if this has been asked before or if I don't provide enough info. I'm having an issue and I hope someone can help me understand what is happening. I have a FreeBSD server that is running Samba. When users access directories that store lots of files (3k or so), access to the files is very slow. I'm guessing that it is a file system issue. I'd like to troubleshoot the problem but I don't know where to start. (In the mean time I have just told the users to create sub-directories and sort their files). Is there anything that I can do to make things run faster? Do you have UFS_DIRHASH in your kernel? If you don't, or if you added it after the disk was already populated, you'll see the benefits if you dump, wipe and restore the disk. Also try updating to 5.3, which is much better than the 5.2.1 developers preview release. Kris pgpwTIr25HwIn.pgp Description: PGP signature
Re: Slow directory access with lots of files
On Mon, 17 Jan 2005 11:13:43 -0800 Kris Kennaway [EMAIL PROTECTED] wrote: Hi, Do you have UFS_DIRHASH in your kernel? If you don't, or if you added it after the disk was already populated, you'll see the benefits if you dump, wipe and restore the disk. Could you elaborate on that? My impression has always been that dirhash does all its magic in memory, without persistant data stored on disk. Cheers, -- Miguel Mendez [EMAIL PROTECTED] http://www.energyhq.es.eu.org PGP Key: 0xDC8514F1 pgpHMiM21Thq3.pgp Description: PGP signature
Re: Slow directory access with lots of files
On Mon, Jan 17, 2005 at 09:34:25PM +0100, Miguel Mendez wrote: On Mon, 17 Jan 2005 11:13:43 -0800 Kris Kennaway [EMAIL PROTECTED] wrote: Hi, Do you have UFS_DIRHASH in your kernel? If you don't, or if you added it after the disk was already populated, you'll see the benefits if you dump, wipe and restore the disk. Could you elaborate on that? My impression has always been that dirhash does all its magic in memory, without persistant data stored on disk. No. It's an optimized method for laying out the data on disk. I think you're confusing it with softupdates, but that still doesn't work entirely that way. Kris pgpB0fK4iP8aA.pgp Description: PGP signature
Re: Slow directory access with lots of files
Kris Kennaway wrote: On Mon, Jan 17, 2005 at 11:09:52AM -0800, Totem wrote: Hello, I'm new to this list so please excuse me if this has been asked before or if I don't provide enough info. I'm having an issue and I hope someone can help me understand what is happening. I have a FreeBSD server that is running Samba. When users access directories that store lots of files (3k or so), access to the files is very slow. I'm guessing that it is a file system issue. I'd like to troubleshoot the problem but I don't know where to start. (In the mean time I have just told the users to create sub-directories and sort their files). Is there anything that I can do to make things run faster? Do you have UFS_DIRHASH in your kernel? If you don't, or if you added it after the disk was already populated, you'll see the benefits if you dump, wipe and restore the disk. Also try updating to 5.3, which is much better than the 5.2.1 developers preview release. Kris I will upgrade to 5.3 in the next few days and I will investigate how to use UFS_DIRHASH. Thanks for the help. Totem ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Slow directory access with lots of files
On Mon, Jan 17, 2005 at 02:49:11PM -0800, Kris Kennaway wrote: On Mon, Jan 17, 2005 at 09:34:25PM +0100, Miguel Mendez wrote: On Mon, 17 Jan 2005 11:13:43 -0800 Kris Kennaway [EMAIL PROTECTED] wrote: Hi, Do you have UFS_DIRHASH in your kernel? If you don't, or if you added it after the disk was already populated, you'll see the benefits if you dump, wipe and restore the disk. Could you elaborate on that? My impression has always been that dirhash does all its magic in memory, without persistant data stored on disk. No. It's an optimized method for laying out the data on disk. I think you're confusing it with softupdates, but that still doesn't work entirely that way. No, this time it is you who are confused. UFS_DIRHASH does not affect the layout on disk at all as far as I can tell. You are probably confusing it with the dirpref changes that were made back in 2001 at approximately the same time as UFS_DIRHASH was added. -- Insert your favourite quote here. Erik Trulsson [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Slow directory access with lots of files
On Tue, Jan 18, 2005 at 01:32:10AM +0100, Erik Trulsson wrote: On Mon, Jan 17, 2005 at 02:49:11PM -0800, Kris Kennaway wrote: On Mon, Jan 17, 2005 at 09:34:25PM +0100, Miguel Mendez wrote: On Mon, 17 Jan 2005 11:13:43 -0800 Kris Kennaway [EMAIL PROTECTED] wrote: Hi, Do you have UFS_DIRHASH in your kernel? If you don't, or if you added it after the disk was already populated, you'll see the benefits if you dump, wipe and restore the disk. Could you elaborate on that? My impression has always been that dirhash does all its magic in memory, without persistant data stored on disk. No. It's an optimized method for laying out the data on disk. I think you're confusing it with softupdates, but that still doesn't work entirely that way. No, this time it is you who are confused. UFS_DIRHASH does not affect the layout on disk at all as far as I can tell. You are probably confusing it with the dirpref changes that were made back in 2001 at approximately the same time as UFS_DIRHASH was added. Sorry, you're right! Kris pgpsZhb9bqFJq.pgp Description: PGP signature
