And compile your own top, Sun and Sunfreeare builds are just a little flaky IMO
Robert Banniza wrote:
I don't think prstat is reporting incorrectly. To verify, you could always use 'top'. Just be aware that top's statistics are not measured the same way as prstat. Just to make sure you have two procs in that box, run 'psrinfo -v' and see if you have two procs. Is there a chance you only have one proc on that machine? Another thought is to maybe use cputrack and cpustat to see what is going on.
Robert
On Wed, Oct 06, 2004 at 01:54:54AM +0100, Stungo, Jamie wrote:
No, hadn't tried that... thanks for the suggestion. Many calls to read() to get the next username logging in. It may be that the high CPU load is due to the pattern matching going on inbetween these calls.
I'd like to understand why only one CPU shows at full load and the other idling. Is it prstat reporting incorrectly?
JS
-----Original Message-----
From: Robert Banniza [mailto:[EMAIL PROTECTED]
Sent: Wed 10/6/2004 1:41 AM
To: [EMAIL PROTECTED]
Cc:
Subject: Re: 0.9.3 Solaris performance problem
Have you tried putting the process under truss (truss -fp <PID>) to see
what kind of system calls are being made by radius? This may give you an
idea of what is going on.
Robert
On Wed, Oct 06, 2004 at 01:33:34AM +0100, Stungo, Jamie wrote:
-----Original Message----- From: Kostas Kalevras [mailto:[EMAIL PROTECTED] Sent: Tue 10/5/2004 11:50 PM To: [EMAIL PROTECTED] Cc: Subject: Re: 0.9.3 Solaris performance problem On Tue, 5 Oct 2004, Stungo, Jamie wrote:
Hi all,
We are experiencing some unexpected behaviour of freeradius on our Solaris 9 platform. We use two V240 dual processor SPARC machines, LDAP back-end, flat file accounting. I have heavily indexed the directory and it seems lightning fast, slapd is running at 0.2% most of the time, yet radiusd chews 95+% of CPU0 and I have to re-nice the process to get a workable shell! This is on both machines. As I understand it we can't spread the load across both CPUs?
Freeradius is multithreaded, so the load sould be spread across both CPUs. >What type is the CPU usage, kernel/user/io?
CPU states: 0.0% idle, 14.6% user, 85.4% kernel, 0.0% iowait, 0.0% swap
I don't believe that the problem is caused by the number of lookups as it was running at fairly low loads (with 10k subs) until we recently added another couple of thousand (who match in the users file instead of dropping through to the LDAP).
I really don't understand this paragraph. What do you mean by subs? Couple thousand of what? Match in the users file?
Subs = subscribers/users. Approximately half the users would do two matches in the users file, the rest just one.
Our users file has about 130 DEFAULT matches (total) as follows:
That's plenty of DEFAULT entries. Remember that for each request the server might try matching 130 entries. Make sure you have the best matching entries first. Though i would not bet my money on that being the >root cause.
It's a good point, I have moved around some entries and I believe this has slightly improved things, however you're right and this is not the main problem.
DEFAULT Suffix == "@subdomain.provider.com", NAS-IP-Address == 10.0.0.1, Auth-Type := Accept Service-Type = Framed, Framed-Protocol = PPP, ERX-Virtual-Router-Name = PROVIDER1, Tunnel-Type = L2TP, Tunnel-Medium-Type = IP, ERX-Tunnel-Password = xxxxxx, Tunnel-Client-Endpoint = 172.X.X.X, Tunnel-Server-Endpoint = 172.X.X.Y, Tunnel-Assignment-Id = xxxx, Tunnel-Client-Auth-Id = blahblah, Tunnel-Server-Auth-Id = blehbleh
DEFAULT Suffix == "@subdomain.provider.com", NAS-IP-Address == 10.0.0.2, Autz-Type := WholesaleLDAP, Auth-Type := Accept Service-Type = Framed, Framed-Protocol = PPP, ERX-Virtual-Router-Name = PROVIDER1, Tunnel-Type = L2TP, Tunnel-Medium-Type = IP, ERX-Tunnel-Password = xxxxxx, Tunnel-Client-Endpoint = 172.X.X.X, Tunnel-Server-Endpoint = 172.X.X.Y, Tunnel-Assignment-Id = xxxxx, Tunnel-Client-Auth-Id = blahblah, Tunnel-Server-Auth-Id = blehbleh
We get a lot of "Unresponsive child" and "Dropping conflicting packet" errors in our radius log, as well as the max number of threads hitting its ceiling (128). Suggestions for a reasonable figure for this for our hardware platform would be helpful to know. It seems to hit its roof at around 250. I'm not sure whether better performance would be gained from allowing it to peak or to keep it low.
Could you post your authorize,authenticate,session and accounting sections? >Do you check for double logins?
We do check for double login checks by means of an LDAP lookup (WholesaleLDAP) on about half the users.
authorize { # # The preprocess module takes care of sanitizing some bizarre # attributes in the request, and turning them into attributes # which are more standard. # # It takes care of processing the 'raddb/hints' and the # 'raddb/huntgroups' files. # # It also adds a Client-IP-Address attribute to the request. preprocess
#
# If you want to have a log of authentication requests,
# un-comment the following line, and the 'detail auth_log'
# section, above.
auth_log
#
# The chap module will set 'Auth-Type := CHAP' if we are
# handling a CHAP request and Auth-Type has not already been set
chap
mschap
# attr_filter
# # This module takes care of EAP-MD5, EAP-TLS, and EAP-LEAP # authentication. #eap
# # If you have a Cisco SIP server authenticating against # FreeRADIUS, uncomment the following line. # digest
# # Look for IPASS style 'realm/', and if not found, look for # '@realm', and decide whether or not to proxy, based on # that. # realmslash suffix
# # If you are using /etc/smbpasswd, and are also doing # mschap authentication, the un-comment this line, and # configure the 'etc_smbpasswd' module, above. # etc_smbpasswd
# # If the users are logging in with an MS-CHAP-Challenge # attribute for authentication, the mschap module will find # the MS-CHAP-Challenge attribute, and add 'Auth-Type := MS-CHAP' # to the request, which will cause the server to then use # the mschap module for authentication. # moved mschap up # mschap
# The ldap module will only be used if the Autz-Type:=LDAP is set in the users
# files. This will in turn set the Auth-Type:=LDAP if it has not already # been set
autztype DirectLDAP {
direct
}
autztype WholesaleLDAP { wholesale }
# # Read the 'users' file files
# daily }
authenticate { # # PAP authentication, when a back-end database listed # in the 'authorize' section supplies a password. The # password can be clear-text, or encrypted. Auth-Type PAP { pap }
# # Most people want CHAP authentication # A back-end database listed in the 'authorize' section # MUST supply a CLEAR TEXT password. Encrypted passwords # won't work. Auth-Type CHAP { chap }
# # MSCHAP authentication. Auth-Type MS-CHAP { mschap }
# # If you have a Cisco SIP server authenticating against # FreeRADIUS, uncomment the following line. # digest
# # Pluggable Authentication Modules. # pam
#
# See 'man getpwent' for information on how the 'unix'
# module checks the users password. Note that packets
# containing CHAP-Password attributes CANNOT be authenticated
# against /etc/passwd! See the FAQ for details.
# # unix
# Uncomment it if you want to use ldap for authentication Auth-Type LDAP { direct }
# # Allow EAP authentication. # eap }
session { # radutmp # sql }
accounting { acct_unique detail # daily unix # wtmp file radutmp # sql # sradutmp # main_pool }
The lookups we're doing don't seem particularly CPU intensive... in the one
? case we're matching domain suffix and NAS-IP-Address and building a tunnel, in ? the other the same but doing a quick lookup in addition. From what I've read
What quick lookup? Lookup on where? ldap?
Yes, this is the WholesaleLDAP lookup I'm referring to.
so far, matches like this should be extremely quick to perform, even with a big users file. I'd like to turn to my LDAP as the source of the problem but I really don't believe it's at fault.
Any and all help gratefully received.
Cheers,
Jamie Stungo
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
-- Kostas Kalevras Network Operations Center [EMAIL PROTECTED] National Technical University of Athens, Greece Work Phone: +30 210 7721861 'Go back to the shadow' Gandalf
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
-- Darin Holloway Systems Administrator and Web Developer -- Bluegrass Network, LLC
- List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
