What version of PF is this?
The was an XML parsing related bug fixed in 4.0.5-2. The error in the logs was
exactly the same as what you are seeing.
--
Louis Munro
lmu...@inverse.ca :: www.inverse.ca
+1.514.447.4918 *125 :: +1 (866) 353-6153
Inverse inc. :: Leaders behind SOGo (www.sogo.nu) and PacketFence
(www.packetfence.org)
On 2013-09-05, at 15:09 , Fletcher Haynes <fhay...@willamette.edu> wrote:
> I cleared the radacct (had around 1 million rows) and radacct_log (had around
> 7 million rows) tables, turned off all accounting for our wireless, and now I
> am back to getting this error: Error: rlm_perl: perl_embed:: module =
> /usr/local/pf/raddb//packetfence.pm , func = post_auth exit status= syntax
> error at line 1, column 49, byte 49 at /usr/lib/perl5/XML/Parser.pm line 187.
>
> I restarted packetfence and mysql machines, put all the networks behind PF
> again, and it ran for about 10 minutes without an issue, then freeradius
> silently died. I was watching the process and its threads in top, and they
> all just disappeared. No error in radius.log, nothing in
> webservices_error_log, nothing in the mysql error logs or slow query logs. No
> core dump in /var/crash, nothing in syslog.
>
> mysqld on the database server was running at about 15% when it died, 30%
> memory. This has happened with 1 core, 2, and 4.
>
> I followed all the MySQL Tuner recommendations. Here is my my.cnf:
>
> user = mysql
> pid-file = /var/run/mysqld/mysqld.pid
> socket = /var/run/mysqld/mysqld.sock
> port = 3306
> basedir = /usr
> datadir = /var/lib/mysql
> tmpdir = /tmp
> lc-messages-dir = /usr/share/mysql
> skip-external-locking
> innodb_flush_log_at_trx_commit = 2
> innodb_buffer_pool_size = 2G
> bind-address = 0.0.0.0
> key_buffer = 900M
> max_allowed_packet = 16M
> thread_stack = 192K
> thread_cache_size = 400
> myisam-recover = BACKUP
> max_connections = 400
> wait_timeout = 180
> interactive_timeout = 180
> table_cache = 500
> query_cache_limit = 128M
> query_cache_size = 512M
> log_error = /var/log/mysql/error.log
> log_slow_queries = /var/log/mysql/mysql-slow.log
> expire_logs_days = 10
> max_binlog_size = 100M
> [mysqldump]
> quick
> quote-names
> max_allowed_packet = 16M
>
> I'm at something of a loss at this point.
>
> On Thu, Sep 5, 2013 at 10:14 AM, Tim DeNike <tim.den...@mcc.edu> wrote:
> Sounds stupid. But try less cores. If your host goes into a wait state for
> an unavailable core it can cause pauses.
> Your mileage may vary. ;)
>
> Sent from my iPhone
>
> On Sep 5, 2013, at 10:59 AM, Fletcher Haynes <fhay...@willamette.edu> wrote:
>
>> Not to hijack Dave's thread, but I started having this issue again two days
>> ago. Except this time, I am not getting any DB handle errors. This is all I
>> see in radius.log:
>> Wed Sep 4 20:54:22 2013 : Error: Discarding duplicate request from client
>> 158.104.249.7 port 32768 - ID: 82 due to unfinished request 10895
>>
>> I see a lot of these, then it handles a few more requests and then dies.
>> From what I can tell, this indicates a slow backend DB, but I have it
>> running on a separate VM with 4GB of RAM and 4 cores. MySQL spikes 100%
>> regularly on one CPU, memory is about 70% used. I"ve done all the MySQL
>> Tuning changes in the Admin guide. At this point, I've had to pull our PF
>> from production, as it isn't stable enough.
>>
>> I have two core dumps from freeradius from when it crashes, but I haven't
>> been able to get anything useful out of them. Would anyone be interested in
>> seeing those? I am going to see if I can compile FR from source with
>> --enable-developer so I can more useful info in gdb. I might also switch to
>> RHEL and see if that has anything to do with it.
>>
>>
>> On Thu, Sep 5, 2013 at 6:11 AM, Louis Munro <lmu...@inverse.ca> wrote:
>> Hi Dave,
>> Have you checked for errors in logs/webservices_errors_log and
>> logs/webservices_access_log ?
>>
>> The packetfence module of FreeRADIUS calls on this webservice. The issue
>> could be related to that.
>>
>> Regards,
>> --
>> Louis Munro
>> lmu...@inverse.ca :: www.inverse.ca
>> +1.514.447.4918 *125 :: +1 (866) 353-6153
>> Inverse inc. :: Leaders behind SOGo (www.sogo.nu) and PacketFence
>> (www.packetfence.org)
>>
>> On 2013-09-04, at 13:07 , Derek Wuelfrath <dwuelfr...@inverse.ca> wrote:
>>
>>>> Debug is probably the only way you'll get an answer.
>>>>
>>>> radiusd -X -d /usr/local/pf/raddb > /usr/local/pf/logs/radius.log 2>&1
>>>
>>> Running FreeRADIUS in debug mode will remove the "multithreading" support
>>> which is almost all of the time the cause of the crash (not that
>>> multithreading is buggy in FreeRADIUS but it is contributing to crashing).
>>>
>>> That can certainly help you but that may also "prevent" your FreeRADIUS
>>> from crashing.
>>>
>>> Derek
>>>
>>> --
>>> Derek Wuelfrath
>>> dwuelfr...@inverse.ca :: +1.514.447.4918 (x110) :: www.inverse.ca
>>> Inverse inc. :: Leaders behind SOGo (www.sogo.nu) and PacketFence
>>> (www.packetfence.org)
>>>
>>> On 2013-09-04, at 6:41 AM, Jason Frisvold <xenoph...@godshell.com> wrote:
>>>
>>>> Bulanda, Dave G wrote:
>>>>> Louis,
>>>>>
>>>>> Radius is still stopping. I have an external NMS monitor for the process
>>>>> and alert and then restart the service pfcmd service radius restart (As
>>>>> it is still leaving a pid file.) At least this is minimizing the impact
>>>>> on my users, but I would like to resolve this.
>>>>>
>>>>> What other places/logs can I check?
>>>>
>>>> Well... Not sure if you're going to like this, but I would recommend
>>>> running radiusd with the -X flag and redirecting output to a log file.
>>>> This puts radiusd into debug mode and will provide significantly more
>>>> information on what's happening. From the log you provided, it appears
>>>> that radiusd is just stopping with no log entry indicating what
>>>> happened. Debug is probably the only way you'll get an answer.
>>>>
>>>> radiusd -X -d /usr/local/pf/raddb > /usr/local/pf/logs/radius.log 2>&1
>>>>
>>>> This does run in the foreground, though, so you'll have to keep a
>>>> session open. I suggest using the screen command for this so a timeout
>>>> doesn't kill your session.
>>>>
>>>>> Thank you again for all the help,
>>>>>
>>>>> David Bulanda
>>>>
>>>>
>>>> --
>>>> ---------------------------
>>>> Jason 'XenoPhage' Frisvold
>>>> xenoph...@godshell.com
>>>> ---------------------------
>>>>
>>>> "Any sufficiently advanced magic is indistinguishable from technology.\"
>>>> - Niven's Inverse of Clarke's Third Law
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
>>>> Discover the easy way to master current and previous Microsoft technologies
>>>> and advance your career. Get an incredible 1,500+ hours of step-by-step
>>>> tutorial videos with LearnDevNow. Subscribe today and save!
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk
>>>> _______________________________________________
>>>> PacketFence-users mailing list
>>>> PacketFence-users@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/packetfence-users
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
>>> Discover the easy way to master current and previous Microsoft technologies
>>> and advance your career. Get an incredible 1,500+ hours of step-by-step
>>> tutorial videos with LearnDevNow. Subscribe today and save!
>>> http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> PacketFence-users mailing list
>>> PacketFence-users@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/packetfence-users
>>
>>
>> ------------------------------------------------------------------------------
>> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
>> Discover the easy way to master current and previous Microsoft technologies
>> and advance your career. Get an incredible 1,500+ hours of step-by-step
>> tutorial videos with LearnDevNow. Subscribe today and save!
>> http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
>> _______________________________________________
>> PacketFence-users mailing list
>> PacketFence-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/packetfence-users
>>
>>
>>
>>
>> --
>> Fletcher Haynes <fhay...@willamette.edu>
>> Systems Administrator/Network Services Consultant
>> Willamette Integrated Technology Services
>> Willamette University, Salem, OR
>> Phone: 503.370.6016
>> ------------------------------------------------------------------------------
>> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
>> Discover the easy way to master current and previous Microsoft technologies
>> and advance your career. Get an incredible 1,500+ hours of step-by-step
>> tutorial videos with LearnDevNow. Subscribe today and save!
>> http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
>> _______________________________________________
>> PacketFence-users mailing list
>> PacketFence-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/packetfence-users
>
> ------------------------------------------------------------------------------
> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
> Discover the easy way to master current and previous Microsoft technologies
> and advance your career. Get an incredible 1,500+ hours of step-by-step
> tutorial videos with LearnDevNow. Subscribe today and save!
> http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
> _______________________________________________
> PacketFence-users mailing list
> PacketFence-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/packetfence-users
>
>
>
>
> --
> Fletcher Haynes <fhay...@willamette.edu>
> Systems Administrator/Network Services Consultant
> Willamette Integrated Technology Services
> Willamette University, Salem, OR
> Phone: 503.370.6016
> ------------------------------------------------------------------------------
> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
> Discover the easy way to master current and previous Microsoft technologies
> and advance your career. Get an incredible 1,500+ hours of step-by-step
> tutorial videos with LearnDevNow. Subscribe today and save!
> http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk_______________________________________________
> PacketFence-users mailing list
> PacketFence-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/packetfence-users
------------------------------------------------------------------------------
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58041391&iu=/4140/ostg.clktrk
_______________________________________________
PacketFence-users mailing list
PacketFence-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/packetfence-users
