I cleared the radacct (had around 1 million rows) and radacct_log (had
around 7 million rows) tables, turned off all accounting for our wireless,
and now I am back to getting this error: Error: rlm_perl: perl_embed::
module = /usr/local/pf/raddb//packetfence.pm , func = post_auth exit
status= syntax error at line 1, column 49, byte 49 at
/usr/lib/perl5/XML/Parser.pm line 187.
I restarted packetfence and mysql machines, put all the networks behind PF
again, and it ran for about 10 minutes without an issue, then freeradius
silently died. I was watching the process and its threads in top, and they
all just disappeared. No error in radius.log, nothing in
webservices_error_log, nothing in the mysql error logs or slow query logs.
No core dump in /var/crash, nothing in syslog.
mysqld on the database server was running at about 15% when it died, 30%
memory. This has happened with 1 core, 2, and 4.
I followed all the MySQL Tuner recommendations. Here is my my.cnf:
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
innodb_flush_log_at_trx_commit = 2
innodb_buffer_pool_size = 2G
bind-address = 0.0.0.0
key_buffer = 900M
max_allowed_packet = 16M
thread_stack = 192K
thread_cache_size = 400
myisam-recover = BACKUP
max_connections = 400
wait_timeout = 180
interactive_timeout = 180
table_cache = 500
query_cache_limit = 128M
query_cache_size = 512M
log_error = /var/log/mysql/error.log
log_slow_queries = /var/log/mysql/mysql-slow.log
expire_logs_days = 10
max_binlog_size = 100M
[mysqldump]
quick
quote-names
max_allowed_packet = 16M
I'm at something of a loss at this point.
On Thu, Sep 5, 2013 at 10:14 AM, Tim DeNike <tim.den...@mcc.edu> wrote:
> Sounds stupid. But try less cores. If your host goes into a wait state
> for an unavailable core it can cause pauses.
> Your mileage may vary. ;)
>
> Sent from my iPhone
>
> On Sep 5, 2013, at 10:59 AM, Fletcher Haynes <fhay...@willamette.edu>
> wrote:
>
> Not to hijack Dave's thread, but I started having this issue again two
> days ago. Except this time, I am not getting any DB handle errors. This is
> all I see in radius.log:
> Wed Sep 4 20:54:22 2013 : Error: Discarding duplicate request from client
> 158.104.249.7 port 32768 - ID: 82 due to unfinished request 10895
>
> I see a lot of these, then it handles a few more requests and then dies.
> From what I can tell, this indicates a slow backend DB, but I have it
> running on a separate VM with 4GB of RAM and 4 cores. MySQL spikes 100%
> regularly on one CPU, memory is about 70% used. I"ve done all the MySQL
> Tuning changes in the Admin guide. At this point, I've had to pull our PF
> from production, as it isn't stable enough.
>
> I have two core dumps from freeradius from when it crashes, but I haven't
> been able to get anything useful out of them. Would anyone be interested in
> seeing those? I am going to see if I can compile FR from source with
> --enable-developer so I can more useful info in gdb. I might also switch to
> RHEL and see if that has anything to do with it.
>
>
> On Thu, Sep 5, 2013 at 6:11 AM, Louis Munro <lmu...@inverse.ca> wrote:
>
>> Hi Dave,
>> Have you checked for errors in logs/webservices_errors_log and
>> logs/webservices_access_log ?
>>
>> The packetfence module of FreeRADIUS calls on this webservice. The issue
>> could be related to that.
>>
>> Regards,
>> --
>> Louis Munro
>> lmu...@inverse.ca :: www.inverse.ca
>> +1.514.447.4918 *125 :: +1 (866) 353-6153
>> Inverse inc. :: Leaders behind SOGo (www.sogo.nu) and PacketFence (
>> www.packetfence.org)
>>
>> On 2013-09-04, at 13:07 , Derek Wuelfrath <dwuelfr...@inverse.ca> wrote:
>>
>> Debug is probably the only way you'll get an answer.
>>
>> radiusd -X -d /usr/local/pf/raddb > /usr/local/pf/logs/radius.log 2>&1
>>
>>
>> Running FreeRADIUS in debug mode will remove the "multithreading" support
>> which is almost all of the time the cause of the crash (not that
>> multithreading is buggy in FreeRADIUS but it is contributing to crashing).
>>
>> That can certainly help you but that may also "prevent" your FreeRADIUS
>> from crashing.
>>
>> Derek
>>
>> --
>> Derek Wuelfrath
>> dwuelfr...@inverse.ca :: +1.514.447.4918 (x110) :: www.inverse.ca
>> Inverse inc. :: Leaders behind SOGo (www.sogo.nu) and PacketFence (
>> www.packetfence.org)
>>
>> On 2013-09-04, at 6:41 AM, Jason Frisvold <xenoph...@godshell.com> wrote:
>>
>> Bulanda, Dave G wrote:
>>
>> Louis,
>>
>> Radius is still stopping. I have an external NMS monitor for the process
>> and alert and then restart the service pfcmd service radius restart (As
>> it is still leaving a pid file.) At least this is minimizing the impact
>> on my users, but I would like to resolve this.
>>
>> What other places/logs can I check?
>>
>>
>> Well... Not sure if you're going to like this, but I would recommend
>> running radiusd with the -X flag and redirecting output to a log file.
>> This puts radiusd into debug mode and will provide significantly more
>> information on what's happening. From the log you provided, it appears
>> that radiusd is just stopping with no log entry indicating what
>> happened. Debug is probably the only way you'll get an answer.
>>
>> radiusd -X -d /usr/local/pf/raddb > /usr/local/pf/logs/radius.log 2>&1
>>
>> This does run in the foreground, though, so you'll have to keep a
>> session open. I suggest using the screen command for this so a timeout
>> doesn't kill your session.
>>
>> Thank you again for all the help,
>>
>> David Bulanda
>>
>>
>>
>> --
>> ---------------------------
>> Jason 'XenoPhage' Frisvold
>> xenoph...@godshell.com
>> ---------------------------
>>
>> "Any sufficiently advanced magic is indistinguishable from technology.\"
>> - Niven's Inverse of Clarke's Third Law
>>
>>
>> ------------------------------------------------------------------------------
>> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
>> Discover the easy way to master current and previous Microsoft
>> technologies
>> and advance your career. Get an incredible 1,500+ hours of step-by-step
>> tutorial videos with LearnDevNow. Subscribe today and save!
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk
>> _______________________________________________
>> PacketFence-users mailing list
>> PacketFence-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/packetfence-users
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
>> Discover the easy way to master current and previous Microsoft
>> technologies
>> and advance your career. Get an incredible 1,500+ hours of step-by-step
>> tutorial videos with LearnDevNow. Subscribe today and save!
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk
>> _______________________________________________
>> PacketFence-users mailing list
>> PacketFence-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/packetfence-users
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
>> Discover the easy way to master current and previous Microsoft
>> technologies
>> and advance your career. Get an incredible 1,500+ hours of step-by-step
>> tutorial videos with LearnDevNow. Subscribe today and save!
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
>> _______________________________________________
>> PacketFence-users mailing list
>> PacketFence-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/packetfence-users
>>
>>
>
>
> --
> Fletcher Haynes <fhay...@willamette.edu>
> Systems Administrator/Network Services Consultant
> Willamette Integrated Technology Services
> Willamette University, Salem, OR
> Phone: 503.370.6016
>
>
> ------------------------------------------------------------------------------
> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
> Discover the easy way to master current and previous Microsoft technologies
> and advance your career. Get an incredible 1,500+ hours of step-by-step
> tutorial videos with LearnDevNow. Subscribe today and save!
> http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
>
> _______________________________________________
> PacketFence-users mailing list
> PacketFence-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/packetfence-users
>
>
>
> ------------------------------------------------------------------------------
> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
> Discover the easy way to master current and previous Microsoft technologies
> and advance your career. Get an incredible 1,500+ hours of step-by-step
> tutorial videos with LearnDevNow. Subscribe today and save!
> http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
> _______________________________________________
> PacketFence-users mailing list
> PacketFence-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/packetfence-users
>
>
--
Fletcher Haynes <fhay...@willamette.edu>
Systems Administrator/Network Services Consultant
Willamette Integrated Technology Services
Willamette University, Salem, OR
Phone: 503.370.6016
------------------------------------------------------------------------------
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
_______________________________________________
PacketFence-users mailing list
PacketFence-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/packetfence-users
