Hi Fabrice,
I mean rtml_perl module takes too much time processing requests and drags
radius very slow.
And I see, no need to login but only need to open mgmt_ip:9000. But which
graphics can tell the issue cause ?
Today we did a pressure test with 50 qps (pf+AD authentication) and found the
freeradius in pf crashed every time and the phenomenon was very similar with
the issue we met recently. We tried to adjust below parameters and the result
was always the same: Freeradius crashed in about 2 minutes. First it became
slow and then crashed and restarted and then we met ??No EAP session match
xxxxxx?? and nearly all requests got rejected. Hardly to believe 50 qps can
ddos freeradius...So any configurations suggestions?
We changed below parameters but the result was the same:
Before we change the parameters in radiusd.conf:
1?? max_request_time = 10
2?? cleanup_delay = 5
3?? max_requests = 20000
4?? reject_delay = 1
5?? max_servers = 512 ??
--------
We changed the parameters in radiusd.conf as below??
1?? max_request_time = 20
2?? cleanup_delay = 10
3?? max_requests = 1280000
4?? reject_delay = 1
5?? max_servers = 512
------------------ Original ------------------
From: Fabrice Durand <[email protected]>
Date: ????,2?? 1,2018 07:43
To: Yan <[email protected]>
Subject: Re: [PacketFence-users] All authentication failed with
error"NoEAPsession matching state xxxx"
Hello Yan,
there is no username and password.
Also what is doperl module ?
Fabrice
Le 2018-01-31 ?? 09:20, Yan a ??crit :
Hi Fabrice,
I never logged in graph GUI, what??s the username and password
it used ? I tried admin GUI account but wrong.
BTW it seems there is a global lock in doperl module and this is
the hard bottleneck as per our stress test...
------------------ Original ------------------
From: Fabrice Durand <[email protected]>
Date: ????,1?? 31,2018 22:04
To: Yan <[email protected]>, packetfence-users
<[email protected]>
Subject: Re: [PacketFence-users] All authentication
failed with error "NoEAPsession matching state xxxx"
Hello Yan,
Le 2018-01-31 ?? 00:28, Yan a ??crit :
Hi dear users,
After a whole night??s analysis, we found it??s pf that takes
too much time processing authentication request if the QPS is
too high and hangs all radius requests later and then Aruba AC
meets the radius timeout setting and re-sends the same radius
access request to pf while pf just sent out the first
radius accept packet and then received the same request, it
will response accept for a second time and then delete the
state id, but Aruba AC might has waited for another 5 seconds
and send a radius request for a third time, and this time pf
find no state id match this session and just response
reject...And then more and more reject responses will cause
user re-connect wireless and the QPS is much more...It's bad
circle...
We find pf has below bottlenecks at least to lead to the hang
issue:
1.Mysql query is too slow.
Most of the times it's because you receive too many
accounting packet (try to disable it) or because there too
many IO.
2."curl" keeps calling httpd service and it's very slow.
Where do you see curl ?, Freeradius use the rest
module to talk to the webservice
3."doperl" is too slow.
Not really, it depend how you configured PacketFence,
let's say you have a ldap source but it take 600ms to do a search
then the radius answer will be slow.
4."ntlm_auth" process is too slow.
Because probably the AD is too slow to answer, btw
you can use the NTLM cache for that.
5.A device will try to connect again if radiusd crashes or
restarted or meets its max requests
But we don't find which configuration will solve this issue
yet. Is there any suggestion on how to change configuration to
handle this performance issue ? Or any basic directions on how
to adjust the parameters to handle 200 QPS,500 QPS and 2000 QPS
?
We have setup that handle millions of request per day
and without any issues, check the graph like radius latency and
also have a look at http://mgmt_ip:9000 and try to find
where it take time.
Btw if you want to us to check your setup, you can ask for a
support with inverse and it will be a pleasure to help you.
Regards
Fabrice
Any response is appreciated. Thank you very very much.
-- Fabrice [email protected] :: +1.514.447.4918 (x135) ::
www.inverse.caInverse inc. :: Leaders behind SOGo (http://www.sogo.nu) and
PacketFence (http://packetfence.org)------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
PacketFence-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/packetfence-users
