Re: nss_ldap / top startup
Hi Gavin,
sorry, took some time to test it, but we're currently very busy moving
services to the new machines.
On Wed, Apr 27, 2005 at 08:06:38PM +0100, Gavin Atkinson wrote:
Sorry - even with that patch, I suspect you'll have to either run top with
the -u option, or define RANDOM_PW before recompiling it. If you can, try
both individually, I'd be interested in your finding.
Your patch is just fine, even without defining RANDOM_PW!
I would suggest you file an PR with the patch, it could be like that:
--- CUT HERE ---
#ifndef TOP_NO_NAMELEN
while ((pw = getpwent()) != NULL) {
if (strlen(pw-pw_name) namelength)
namelength = strlen(pw-pw_name);
}
#else
namelength = 15;
#endif
--- CUT HERE ---
In the Makefile there should be something like that:
--- CUT HERE ---
.if defined(TOP_NO_NAMELEN)
CFLAGS+= -DTOP_NO_NAMELEN
.endif
--- CUT HERE ---
If you don't want to file a PR, I would like to ask for permission to
put the changes into a patch and file the PR, I guess there are more
people having lot's of users - and always reading them all and counting
the namelength seems to be eating resources unnecessarily, at leats on
systems with some 10k users.
Thank you,
Oliver
--
| Oliver Brandmueller | Offenbacher Str. 1 | Germany D-14197 Berlin |
| Fon +49-172-3130856 | Fax +49-172-3145027 | WWW: http://the.addict.de/ |
| Ich bin das Internet. Sowahr ich Gott helfe. |
| Eine gewerbliche Nutzung aller enthaltenen Adressen ist nicht gestattet! |
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: nss_ldap / top startup
On Mon, 2005-04-25 at 12:59 +0200, Oliver Brandmueller wrote:
Hi,
I have some servers running running on 5.4-STABLE as of Apr 5th. I use
nss_ldap for a userbase of currently about 24000 accounts (will be
growing to approx 6 in the next weeks). I don't use pam_ldap
currently, because users only need to login by IMAP, POP, SMTP and FTP,
for all of these services daemons are used which natively auth against
the LDAP server.
The more accounts there are in the LDAP directory, the longer the
startup of top takes. With the current userbase top takes about 3-4
seconds to start (on a mostly idle Dual Xeon 2.8GHz with fast disks and
local slapd).
FWIW, I don't think this is related to LDAP as such. I have a machine
bound to NIS with ~19000 entries in the passwd file. Top takes ages to
start up.
The problem is in machine.c - it iterates over every user in the passwd
file to figure out what how many characters longest username may be.
It's nasty and to be honest I think it can/should just be removed.
Try the attached patch just to prove that this is the cause in your case
too.
A while back, there was talk of a FreeBSD libc name cache daemon, but I
can't seem to find any reference to it now (I seem to remember the
website was within .ru, if it helps anyone find it) - though I'm not
sure it would help in this context or even if it's really the correct
way to mask the bug.
Gavin
Index: src/usr.bin/top/machine.c
===
RCS file: /usr/cvs/src/usr.bin/top/machine.c,v
retrieving revision 1.68
diff -u -r1.68 machine.c
--- src/usr.bin/top/machine.c 16 Aug 2004 07:51:22 - 1.68
+++ src/usr.bin/top/machine.c 27 Apr 2005 18:03:09 -
@@ -228,10 +228,13 @@
modelen != sizeof(smpmode))
smpmode = 0;
+#if 0
while ((pw = getpwent()) != NULL) {
if (strlen(pw-pw_name) namelength)
namelength = strlen(pw-pw_name);
}
+#endif
+ namelength = 8;
if (namelength 8)
namelength = 8;
if (smpmode namelength 13)
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: nss_ldap / top startup
On Wed, 27 Apr 2005, Gavin Atkinson wrote: On Mon, 2005-04-25 at 12:59 +0200, Oliver Brandmueller wrote: The more accounts there are in the LDAP directory, the longer the startup of top takes. With the current userbase top takes about 3-4 seconds to start (on a mostly idle Dual Xeon 2.8GHz with fast disks and local slapd). The problem is in machine.c - it iterates over every user in the passwd file to figure out what how many characters longest username may be. It's nasty and to be honest I think it can/should just be removed. Try the attached patch just to prove that this is the cause in your case too. Sorry - even with that patch, I suspect you'll have to either run top with the -u option, or define RANDOM_PW before recompiling it. If you can, try both individually, I'd be interested in your finding. Gavin ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: nss_ldap / top startup
On 4/27/05, Gavin Atkinson [EMAIL PROTECTED] wrote: A while back, there was talk of a FreeBSD libc name cache daemon, but I can't seem to find any reference to it now (I seem to remember the website was within .ru, if it helps anyone find it) - though I'm not sure it would help in this context or even if it's really the correct way to mask the bug. Perhaps you are thinking of sysutils/lookupd and sysutils/lookupd-ldap in the ports tree? I've been using those in conjuction with nss-ldap and I've been happy with the results (though I don't have enough users to stress things the way you folks do). ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: nss_ldap / top startup
Hi, Any ideas, why this is happening? Will I need 10 seconds, when there are 6 accounts in LDAP? :-) Have you set your indexes correctly? This may result in a huge speedup. I use (this is only for about 180 accounts, but NSS works instantly): # Indices to maintain index objectClass eq index cn sub,pres,eq index uid,uidNumber,gidNumber,homeDirectory pres,eq Also, have you tried truss(1) or ktrace(1) on the process? It might help you try to see if the LDAP server is just being slow, or the LDAP client is. -- Rink P.W. Springer- http://rink.nu God, root, what is difference? - Pitr, Userfriendly pgpILhXpgyjhU.pgp Description: PGP signature
Re: nss_ldap / top startup
Hi. On Tue, Apr 26, 2005 at 07:55:37AM +0200, Rink Springer wrote: Any ideas, why this is happening? Will I need 10 seconds, when there are 6 accounts in LDAP? :-) Have you set your indexes correctly? This may result in a huge speedup. I use (this is only for about 180 accounts, but NSS works instantly): Yes, indices and caching are set well and I don't have any problems with the speed of the LDAP server in any way (SMTP, POP, IMAP and so on all work very fast and smooth). Also, have you tried truss(1) or ktrace(1) on the process? It might help you try to see if the LDAP server is just being slow, or the LDAP client is. Well, I wonder why LDAP should be queried (by nss_ldap) at all (with -u used as top option). I did not yet truss/ktrace top, but maybe I should. - Oliver -- | Oliver Brandmueller | Offenbacher Str. 1 | Germany D-14197 Berlin | | Fon +49-172-3130856 | Fax +49-172-3145027 | WWW: http://the.addict.de/ | | Ich bin das Internet. Sowahr ich Gott helfe. | | Eine gewerbliche Nutzung aller enthaltenen Adressen ist nicht gestattet! | pgpYj51z5RvR1.pgp Description: PGP signature
nss_ldap / top startup
Hi, I have some servers running running on 5.4-STABLE as of Apr 5th. I use nss_ldap for a userbase of currently about 24000 accounts (will be growing to approx 6 in the next weeks). I don't use pam_ldap currently, because users only need to login by IMAP, POP, SMTP and FTP, for all of these services daemons are used which natively auth against the LDAP server. The more accounts there are in the LDAP directory, the longer the startup of top takes. With the current userbase top takes about 3-4 seconds to start (on a mostly idle Dual Xeon 2.8GHz with fast disks and local slapd). The startup time is not any different, sometimes I feel (did not try to measure) it's even longer, if I use top -u to not map uids. The running processes are only from a few uids, all the LDAP users usually don't have processes running under thier IDs. Any ideas, why this is happening? Will I need 10 seconds, when there are 6 accounts in LDAP? :-) - Oliver -- | Oliver Brandmueller | Offenbacher Str. 1 | Germany D-14197 Berlin | | Fon +49-172-3130856 | Fax +49-172-3145027 | WWW: http://the.addict.de/ | | Ich bin das Internet. Sowahr ich Gott helfe. | | Eine gewerbliche Nutzung aller enthaltenen Adressen ist nicht gestattet! | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: nss_ldap / top startup
In the last episode (Apr 25), Oliver Brandmueller said: I have some servers running running on 5.4-STABLE as of Apr 5th. I use nss_ldap for a userbase of currently about 24000 accounts (will be growing to approx 6 in the next weeks). I don't use pam_ldap currently, because users only need to login by IMAP, POP, SMTP and FTP, for all of these services daemons are used which natively auth against the LDAP server. The more accounts there are in the LDAP directory, the longer the startup of top takes. With the current userbase top takes about 3-4 seconds to start (on a mostly idle Dual Xeon 2.8GHz with fast disks and local slapd). The startup time is not any different, sometimes I feel (did not try to measure) it's even longer, if I use top -u to not map uids. The running processes are only from a few uids, all the LDAP users usually don't have processes running under thier IDs. You can benchmark top by running time top -d1, which will print one page then immediately exit. Any ideas, why this is happening? Will I need 10 seconds, when there are 6 accounts in LDAP? :-) Try editing /usr/src/usr.bin/top/Makefile, add -DRANDOM_PW, and rebuild. That should probably be the default on FreeBSD anyway. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: nss_ldap / top startup
Hello. On Mon, Apr 25, 2005 at 12:00:23PM -0500, Dan Nelson wrote: You can benchmark top by running time top -d1, which will print one page then immediately exit. Well, it makes really no difference, if I call top with -u or not. Any ideas, why this is happening? Will I need 10 seconds, when there are 6 accounts in LDAP? :-) Try editing /usr/src/usr.bin/top/Makefile, add -DRANDOM_PW, and rebuild. That should probably be the default on FreeBSD anyway. I tried RANDOM_PW and I tried a bigger TOP_TABLE_SIZE, gives me no difference. It really seems to be the case, that the machine reads a lot of stuff from the slapd, when top starts you top and slapd being busy (both are going back to nearly 0 very soon). Well; what information is needed from LDAP, if I call top -u ? It makes no username mapping. ps -aux is by far faster, so I guess it's reading all the information and building it's table, althoug it's later not really used? - Oliver -- | Oliver Brandmueller | Offenbacher Str. 1 | Germany D-14197 Berlin | | Fon +49-172-3130856 | Fax +49-172-3145027 | WWW: http://the.addict.de/ | | Ich bin das Internet. Sowahr ich Gott helfe. | | Eine gewerbliche Nutzung aller enthaltenen Adressen ist nicht gestattet! | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
