In case you want to look at some fancy graphs... You can see when the server was restarted this morning: https://imgur.com/a/BnM3Z
And here are some graphs that show the week. Notice that before 10/2, everything looked very stable. On 10/2 I spent some time trying to get accounting and fingerbank working: https://imgur.com/a/mrEwE On Tue, Oct 6, 2015 at 9:41 AM, Chris Abel <[email protected]> wrote: > # reboot > > A simple restart of packetfence won't fix things. I need to turn of sql > accounting and reboot the server. Then my radius log looks like this: > > Tue Oct 6 09:21:05 2015 : Auth: Login OK: [f4f951e90034] (from client > 10.131.4.18 port 0 cli F4-F9-51-E9-00-34) > Tue Oct 6 09:21:05 2015 : Auth: Login OK: [f4f951e90034] (from client > 10.131.4.10 port 0 cli F4-F9-51-E9-00-34) > Tue Oct 6 09:21:05 2015 : Auth: Login OK: [f4f951e90034] (from client > 10.131.4.13 port 0 cli F4-F9-51-E9-00-34) > Tue Oct 6 09:21:05 2015 : Info: WARNING: Child is hung for request 456 in > component post-auth module packetfence. > Tue Oct 6 09:21:07 2015 : Info: WARNING: Child is hung for request 456 in > component post-auth module packetfence. > Tue Oct 6 09:21:07 2015 : Error: Discarding duplicate request from client > 10.131.4.18 port 56942 - ID: 66 due to unfinished request 458 in component > post-auth module packetfence. > Tue Oct 6 09:21:08 2015 : Error: Discarding duplicate request from client > 10.131.4.10 port 53447 - ID: 4 due to unfinished request 459 in component > post-auth module packetfence. > Tue Oct 6 09:21:08 2015 : Auth: Login OK: [48d705c10339] (from client > 10.128.4.14 port 0 cli 48-D7-05-C1-03-39) > Tue Oct 6 09:21:08 2015 : Error: Discarding duplicate request from client > 10.131.4.13 port 37715 - ID: 16 due to unfinished request 460 in component > post-auth module packetfence. > Tue Oct 6 09:21:08 2015 : Auth: Login OK: [48d705c10339] (from client > 10.128.4.16 port 0 cli 48-D7-05-C1-03-39) > Tue Oct 6 09:21:08 2015 : Auth: Login OK: [48d705c10339] (from client > 10.128.4.12 port 0 cli 48-D7-05-C1-03-39) > > > I still get some child is hung messages, not sure if that's just left over > from before the reboot or that is what is actually causing my system to > stall. > > On Tue, Oct 6, 2015 at 9:32 AM, Louis Munro <[email protected]> wrote: > >> How exactly did you reboot that server? >> >> -- >> Louis Munro >> [email protected] :: www.inverse.ca >> +1.514.447.4918 x125 :: +1 (866) 353-6153 x125 >> Inverse inc. :: Leaders behind SOGo (www.sogo.nu) and PacketFence ( >> www.packetfence.org) >> >> On Oct 6, 2015, at 9:10 , Chris Abel <[email protected]> wrote: >> >> Commented out sql in /pf/conf/radiusd/packetfence and things seem stable >> again. It definitely has something to do with accounting and the connection >> to mysql. Found these in my syslog after rebooting the server. Anything to >> be concerned with? >> >> # cat /var/log/syslog | grep mysql >> Oct 6 08:48:20 packetfence mysqld_safe: Starting mysqld daemon with >> databases from /var/lib/mysql >> Oct 6 08:48:20 packetfence mysqld: 151006 8:48:20 [Warning] Using >> unique option prefix key_buffer instead of key_buffer_size is deprecated >> and will be removed in a future release. Please use the full name instead. >> Oct 6 08:48:20 packetfence mysqld: 151006 8:48:20 [Note] >> /usr/sbin/mysqld (mysqld 5.5.44-0+deb7u1) starting as process 2717 ... >> Oct 6 08:48:20 packetfence mysqld: 151006 8:48:20 [Warning] Using >> unique option prefix myisam-recover instead of myisam-recover-options is >> deprecated and will be removed in a future release. Please use the full >> name instead. >> Oct 6 08:48:20 packetfence mysqld: 151006 8:48:20 [Note] Plugin >> 'FEDERATED' is disabled. >> Oct 6 08:48:20 packetfence mysqld: 151006 8:48:20 InnoDB: The InnoDB >> memory heap is disabled >> Oct 6 08:48:20 packetfence mysqld: 151006 8:48:20 InnoDB: Mutexes and >> rw_locks use GCC atomic builtins >> Oct 6 08:48:20 packetfence mysqld: 151006 8:48:20 InnoDB: Compressed >> tables use zlib 1.2.7 >> Oct 6 08:48:20 packetfence mysqld: 151006 8:48:20 InnoDB: Using Linux >> native AIO >> Oct 6 08:48:21 packetfence mysqld: 151006 8:48:21 InnoDB: Initializing >> buffer pool, size = 128.0M >> Oct 6 08:48:21 packetfence mysqld: 151006 8:48:21 InnoDB: Completed >> initialization of buffer pool >> Oct 6 08:48:22 packetfence mysqld: 151006 8:48:22 InnoDB: highest >> supported file format is Barracuda. >> Oct 6 08:48:22 packetfence mysqld: InnoDB: Log scan progressed past the >> checkpoint lsn 11427549867 >> Oct 6 08:48:22 packetfence mysqld: 151006 8:48:22 InnoDB: Database was >> not shut down normally! >> Oct 6 08:48:22 packetfence mysqld: InnoDB: Starting crash recovery. >> Oct 6 08:48:22 packetfence mysqld: InnoDB: Reading tablespace >> information from the .ibd files... >> Oct 6 08:48:22 packetfence mysqld: InnoDB: Restoring possible >> half-written data pages from the doublewrite >> Oct 6 08:48:22 packetfence mysqld: InnoDB: buffer... >> Oct 6 08:48:22 packetfence mysqld: InnoDB: Doing recovery: scanned up to >> log sequence number 11427561597 >> Oct 6 08:48:23 packetfence mysqld: 151006 8:48:23 InnoDB: Starting an >> apply batch of log records to the database... >> Oct 6 08:48:23 packetfence mysqld: InnoDB: Progress in percents: 5 6 7 8 >> 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 >> 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 >> 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 >> 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 >> Oct 6 08:48:23 packetfence mysqld: InnoDB: Apply batch completed >> Oct 6 08:48:23 packetfence mysqld: 151006 8:48:23 InnoDB: Waiting for >> the background threads to start >> Oct 6 08:48:24 packetfence mysqld: 151006 8:48:24 InnoDB: 5.5.44 >> started; log sequence number 11427561597 >> Oct 6 08:48:24 packetfence mysqld: 151006 8:48:24 [Note] Server >> hostname (bind-address): '127.0.0.1'; port: 3306 >> Oct 6 08:48:24 packetfence mysqld: 151006 8:48:24 [Note] - >> '127.0.0.1' resolves to '127.0.0.1'; >> Oct 6 08:48:24 packetfence mysqld: 151006 8:48:24 [Note] Server socket >> created on IP: '127.0.0.1'. >> Oct 6 08:48:24 packetfence mysqld: 151006 8:48:24 [ERROR] >> /usr/sbin/mysqld: Table './mysql/user' is marked as crashed and should be >> repaired >> Oct 6 08:48:24 packetfence mysqld: 151006 8:48:24 [Warning] Checking >> table: './mysql/user' >> Oct 6 08:48:24 packetfence mysqld: 151006 8:48:24 [ERROR] 1 client is >> using or hasn't closed the table properly >> Oct 6 08:48:24 packetfence mysqld: 151006 8:48:24 [ERROR] >> /usr/sbin/mysqld: Table './mysql/db' is marked as crashed and should be >> repaired >> Oct 6 08:48:24 packetfence mysqld: 151006 8:48:24 [Warning] Checking >> table: './mysql/db' >> Oct 6 08:48:24 packetfence mysqld: 151006 8:48:24 [ERROR] 1 client is >> using or hasn't closed the table properly >> Oct 6 08:48:24 packetfence mysqld: 151006 8:48:24 [Note] Event >> Scheduler: Loaded 0 events >> Oct 6 08:48:24 packetfence mysqld: 151006 8:48:24 [Note] >> /usr/sbin/mysqld: ready for connections. >> Oct 6 08:48:24 packetfence mysqld: Version: '5.5.44-0+deb7u1' socket: >> '/var/run/mysqld/mysqld.sock' port: 3306 (Debian) >> Oct 6 08:48:25 packetfence /etc/mysql/debian-start[2883]: Upgrading >> MySQL tables if necessary. >> Oct 6 08:48:25 packetfence /etc/mysql/debian-start[2887]: >> /usr/bin/mysql_upgrade: the '--basedir' option is always ignored >> Oct 6 08:48:25 packetfence /etc/mysql/debian-start[2887]: Looking for >> 'mysql' as: /usr/bin/mysql >> Oct 6 08:48:25 packetfence /etc/mysql/debian-start[2887]: Looking for >> 'mysqlcheck' as: /usr/bin/mysqlcheck >> Oct 6 08:48:25 packetfence /etc/mysql/debian-start[2887]: This >> installation of MySQL is already upgraded to 5.5.44, use --force if you >> still need to run mysql_upgrade >> Oct 6 08:48:25 packetfence /etc/mysql/debian-start[2902]: Checking for >> insecure root accounts. >> Oct 6 08:48:25 packetfence /etc/mysql/debian-start[2907]: Triggering >> myisam-recover for all MyISAM tables >> Oct 6 08:48:25 packetfence mysqld: 151006 8:48:25 [ERROR] >> /usr/sbin/mysqld: Table './pf_graphite/account_profile' is marked as >> crashed and should be repaired >> Oct 6 08:48:25 packetfence mysqld: 151006 8:48:25 [Warning] Checking >> table: './pf_graphite/account_profile' >> Oct 6 08:48:25 packetfence mysqld: 151006 8:48:25 [ERROR] >> /usr/sbin/mysqld: Table './pf_graphite/auth_user' is marked as crashed and >> should be repaired >> Oct 6 08:48:25 packetfence mysqld: 151006 8:48:25 [Warning] Checking >> table: './pf_graphite/auth_user' >> >> On Tue, Oct 6, 2015 at 8:49 AM, Chris Abel <[email protected]> >> wrote: >> >>> So this morning the wireless was down again and my radius showed "Child >>> is hung" messages. My system seems that it is under unnecessary load. How >>> can I find out what is causing this? >>> >>> On Mon, Oct 5, 2015 at 3:14 PM, Chris Abel <[email protected]> >>> wrote: >>> >>>> Shows accounting traffic: >>>> >>>> IP 10.131.4.17.52402 > 10.128.0.66.1813: RADIUS, Accounting Request >>>> (4), id: 0xae length: 182 >>>> IP 10.128.0.66.1813 > 10.131.4.17.52402: RADIUS, Accounting Response >>>> (5), id: 0xae length: 59 >>>> IP 10.131.4.14.43804 > 10.128.0.66.1813: RADIUS, Accounting Request >>>> (4), id: 0xdc length: 182 >>>> IP 10.128.0.66.1813 > 10.131.4.14.43804: RADIUS, Accounting Response >>>> (5), id: 0xdc length: 59 >>>> IP 10.128.4.17.55959 > 10.128.0.66.1813: RADIUS, Accounting Request >>>> (4), id: 0x04 length: 182 >>>> IP 10.128.0.66.1813 > 10.128.4.17.55959: RADIUS, Accounting Response >>>> (5), id: 0x04 length: 59 >>>> IP 10.128.4.17.55959 > 10.128.0.66.1813: RADIUS, Accounting Request >>>> (4), id: 0x05 length: 182 >>>> IP 10.128.0.66.1813 > 10.128.4.17.55959: RADIUS, Accounting Response >>>> (5), id: 0x05 length: 59 >>>> >>>> >>>> >>>> On Mon, Oct 5, 2015 at 2:57 PM, Louis Munro <[email protected]> wrote: >>>> >>>>> >>>>> >>>>> On Oct 5, 2015, at 13:40 , Chris Abel <[email protected]> >>>>> wrote: >>>>> >>>>> Accounting is still not working after setting the port to 1813. Does >>>>> it take a while to collect data? >>>>> >>>>> >>>>> >>>>> It should not. >>>>> Is anything coming in on port 1813? >>>>> >>>>> What does this show? >>>>> >>>>> # tcpdump -iany -tnl port 1813 >>>>> >>>>> >>>>> Regards, >>>>> -- >>>>> Louis Munro >>>>> [email protected] :: www.inverse.ca >>>>> +1.514.447.4918 x125 :: +1 (866) 353-6153 x125 >>>>> Inverse inc. :: Leaders behind SOGo (www.sogo.nu) and PacketFence ( >>>>> www.packetfence.org) >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> _______________________________________________ >>>>> PacketFence-users mailing list >>>>> [email protected] >>>>> https://lists.sourceforge.net/lists/listinfo/packetfence-users >>>>> >>>>> >>>> >>>> >>>> -- >>>> Chris Abel >>>> Systems and Network Administrator >>>> Wildwood Programs >>>> 2995 Curry Road Extension >>>> Schenectady, NY 12303 >>>> 518-836-2341 >>>> >>> >>> >>> >>> -- >>> Chris Abel >>> Systems and Network Administrator >>> Wildwood Programs >>> 2995 Curry Road Extension >>> Schenectady, NY 12303 >>> 518-836-2341 >>> >> >> >> >> -- >> Chris Abel >> Systems and Network Administrator >> Wildwood Programs >> 2995 Curry Road Extension >> Schenectady, NY 12303 >> 518-836-2341 >> >> IMPORTANT NOTICE: This message and any attachments are solely for the >> intended recipient and may contain confidential information, which is, or >> may be, legally privileged or otherwise protected by law from further >> disclosure. If you are not the intended recipient, any disclosure, copying, >> use, or distribution of the information included in this email and any >> attachments is prohibited. If you have received this communication in >> error, please notify the sender by reply email and immediately and >> permanently delete this email and any attachments. >> ------------------------------------------------------------------------------ >> _______________________________________________ >> PacketFence-users mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/packetfence-users >> >> >> >> >> ------------------------------------------------------------------------------ >> >> _______________________________________________ >> PacketFence-users mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/packetfence-users >> >> > > > -- > Chris Abel > Systems and Network Administrator > Wildwood Programs > 2995 Curry Road Extension > Schenectady, NY 12303 > 518-836-2341 > -- Chris Abel Systems and Network Administrator Wildwood Programs 2995 Curry Road Extension Schenectady, NY 12303 518-836-2341 -- IMPORTANT NOTICE: This message and any attachments are solely for the intended recipient and may contain confidential information, which is, or may be, legally privileged or otherwise protected by law from further disclosure. If you are not the intended recipient, any disclosure, copying, use, or distribution of the information included in this email and any attachments is prohibited. If you have received this communication in error, please notify the sender by reply email and immediately and permanently delete this email and any attachments.
------------------------------------------------------------------------------
_______________________________________________ PacketFence-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/packetfence-users
