No... isn't that a GUI? This is a colo'ed server running a prepaid calling card app.
Cheers, j On Wed, 19 Nov 2008, Steve Totaro wrote: > Are you using NetworkManager? > > On Wed, Nov 19, 2008 at 8:29 PM, Jeff LaCoursiere <[EMAIL PROTECTED]> wrote: >> >> Happy for all suggestions, of course! No offense intended with my reply. >> Not sure what you are trying to get at with init.d, but here you go: >> >> ast% ls /etc/init.d >> /etc/init.d@ >> >> Am guessing you expected a bit more than that, so allow me to assume what >> you are looking for (and sum up the state of the discussion): >> >> ast% ls -ltr /etc/init.d/iptables >> -rwxr-xr-x 1 root root 7135 Nov 11 2004 /etc/init.d/iptables* >> ast% >> >> This is of course the script that loads and unloads the kernel modules >> associated with iptables, and was run as "iptables stop" on November 3rd >> which caused the problem in discussion. One of the lines in this script >> does a "modprobe -r ipt_state" which hung. Apparently the actual module >> which hung while unloading is ip_conntrack: >> >> ast% cat /proc/modules | head >> ip_conntrack 45573 0 - Unloading 0xf8945000 >> [snip] >> >> Some bug in this module must be in an infinite loop in kernel space, as >> the process eats 100% of the CPU even when reniced to +19. Because of >> this the modprobe process cannot be killed and won't respond to >> interrupts. >> >> An attempt at getting a kernel stack trace failed, which is extremely >> unfortunate :( Cool to learn about /proc/sysrq-trigger, though! >> >> Did I miss anything? >> >> Cheers, >> >> j >> >> On Wed, 19 Nov 2008, Steve Totaro wrote: >> >>> I was not implying that you upgrade anything but iptables. >>> >>> What is the output of "ls /etc/init.d/" >>> >>> On Wed, Nov 19, 2008 at 8:02 PM, Jeff LaCoursiere <[EMAIL PROTECTED]> wrote: >>>> >>>> Hi Steve, >>>> >>>> [EMAIL PROTECTED] ~]# ls -ltr /etc/init.d >>>> lrwxrwxrwx 1 root root 11 Nov 29 2007 /etc/init.d -> rc.d/init.d >>>> [EMAIL PROTECTED] ~]# >>>> >>>> Although I agree that updating the kernel et all would be a good idea, the >>>> whole point is to keep the machine running for 19 more days without the >>>> rogue process interfering with my voice quality. If I cannot unload the >>>> module or otherwise interrupt the process which is currently spinning in >>>> kernel space, no upgrade will be possible. I am quite sure that rebooting >>>> will fix this problem, but the "puzzle" was to fix it without doing so... >>>> >>>> Cheers, >>>> >>>> j >>>> >>>> On Wed, 19 Nov 2008, Steve Totaro wrote: >>>> >>>>> Well then use whatever package manager you have. Apt-get I assume. >>>>> Maybe that might help. >>>>> >>>>> What do you get with "#ls -ltr /etc/init.d"? >>>>> -- >>>>> Thanks, >>>>> Steve Totaro >>>>> +18887771888 (Toll Free) >>>>> +12409381212 (Cell) >>>>> +12024369784 (Skype) >>>>> >>>>> On Wed, Nov 19, 2008 at 7:19 PM, Jeff LaCoursiere <[EMAIL PROTECTED]> >>>>> wrote: >>>>>> >>>>>> Its not Centos - there is no 'yum'. >>>>>> "service iptables stop" is what >>>>>> produced the hanging process in the first place - I think my big problem >>>>>> here is that a kernel module is broken, and there is no way to stop it, >>>>>> and there seems to be no way to unload it (in fact it is hung trying to >>>>>> do >>>>>> just that). >>>>>> >>>>>> Thanks for the suggestions, though! >>>>>> >>>>>> j >>>>>> >>>>>> On Wed, 19 Nov 2008, Steve Totaro wrote: >>>>>> >>>>>>> YUM update? "service iptables stop" "service iptables start"? >>>>>>> >>>>>>> On Wed, Nov 19, 2008 at 4:46 PM, Jeff LaCoursiere <[EMAIL PROTECTED]> >>>>>>> wrote: >>>>>>>> >>>>>>>> Hmm, I am more of a BSD guy I guess. I would expect a pipe to show a >>>>>>>> 'p' >>>>>>>> in a long ls. This is interesting though: >>>>>>>> >>>>>>>> [EMAIL PROTECTED] init.d]# cat /proc/modules | head >>>>>>>> ip_conntrack 45573 0 - Unloading 0xf8945000 >>>>>>>> [EMAIL PROTECTED] init.d]# rmmod -f ip_conntrack >>>>>>>> ERROR: Removing 'ip_conntrack': Device or resource busy >>>>>>>> >>>>>>>> (sigh) >>>>>>>> >>>>>>>> I am pretty sure ip_conntrack is part of the iptables stuff... >>>>>>>> >>>>>>>> j >>>>>>>> >>>>>>>> On Wed, 19 Nov 2008, Danny Nicholas wrote: >>>>>>>> >>>>>>>>> /proc/modules is a pipe >>>>>>>>> You can see what is in there by type cat /proc/modules|more >>>>>>>>> >>>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: [EMAIL PROTECTED] >>>>>>>>> [mailto:[EMAIL PROTECTED] On Behalf Of Jeff >>>>>>>>> LaCoursiere >>>>>>>>> Sent: Wednesday, November 19, 2008 2:47 PM >>>>>>>>> To: Asterisk Users Mailing List - Non-Commercial Discussion >>>>>>>>> Subject: Re: [asterisk-users] puzzle >>>>>>>>> >>>>>>>>> >>>>>>>>> A good idea! The modprobe command is actually in the ps below - it is >>>>>>>>> part of the /etc/init.d/iptables script, and apparently was trying to >>>>>>>>> remove the ipt_state module. The result, however: >>>>>>>>> >>>>>>>>> [EMAIL PROTECTED] init.d]# rmmod ipt_state >>>>>>>>> ERROR: Module ipt_state does not exist in /proc/modules >>>>>>>>> >>>>>>>>> (sigh). In fact /proc/modules is empty. >>>>>>>>> >>>>>>>>> [EMAIL PROTECTED] init.d]# ls -ltr /proc/modules >>>>>>>>> -r--r--r-- 1 root root 0 Nov 19 14:46 /proc/modules >>>>>>>>> >>>>>>>>> j >>>>>>>>> >>>>>>>>> On Wed, 19 Nov 2008, Danny Nicholas wrote: >>>>>>>>> >>>>>>>>>> Your could try this >>>>>>>>>> History|grep modprobe >>>>>>>>>> Rmmod XXX where xxx is the parameter from the history|grep modprobe. >>>>>>>>>> This of course assumes that the command is in your last 1000 >>>>>>>>>> commands. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -----Original Message----- >>>>>>>>>> From: [EMAIL PROTECTED] >>>>>>>>>> [mailto:[EMAIL PROTECTED] On Behalf Of Jeff >>>>>>>>>> LaCoursiere >>>>>>>>>> Sent: Wednesday, November 19, 2008 2:20 PM >>>>>>>>>> To: Asterisk Users Mailing List - Non-Commercial Discussion >>>>>>>>>> Subject: Re: [asterisk-users] puzzle >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yes, the second 'ps' below showed the parent to be '1' (init), which >>>>>>>>>> means >>>>>>>>>> its real parent died already. >>>>>>>>>> >>>>>>>>>> Any attempt to flush the iptables hangs :( >>>>>>>>>> >>>>>>>>>> j >>>>>>>>>> >>>>>>>>>> On Wed, 19 Nov 2008, Danny Nicholas wrote: >>>>>>>>>> >>>>>>>>>>> Have you done a ps -elf to see if the process has a parent that is >>>>>>>>>>> re-launching or preserving it? >>>>>>>>>>> >>>>>>>>>>> -----Original Message----- >>>>>>>>>>> From: [EMAIL PROTECTED] >>>>>>>>>>> [mailto:[EMAIL PROTECTED] On Behalf Of Jeff >>>>>>>>>>> LaCoursiere >>>>>>>>>>> Sent: Wednesday, November 19, 2008 1:58 PM >>>>>>>>>>> To: asterisk-users@lists.digium.com >>>>>>>>>>> Subject: [asterisk-users] puzzle >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Sorry again for the only marginal relation to asterisk, but the >>>>>>>>>>> issue >>>>>>>>> does >>>>>>>>>>> affect the voice performance I am experiencing, so I am soothing my >>>>>>>>>>> guilt >>>>>>>>>>> with that. >>>>>>>>>>> >>>>>>>>>>> Bet you don't see this every day: >>>>>>>>>>> >>>>>>>>>>> ast% uptime >>>>>>>>>>> 13:48:08 up 981 days, 18:29, 1 user, load average: 1.08, 1.02, >>>>>>>>>>> 1.01 >>>>>>>>>>> ast% >>>>>>>>>>> >>>>>>>>>>> I *REALLY* want this machine to see 1000 days uptime, if for nothing >>>>>>>>> other >>>>>>>>>>> than bragging rights. Its been through mysql and asterisk >>>>>>>>>>> upgrades, a >>>>>>>>>>> horrible hacking nightmare that very nearly made me reboot, and >>>>>>>>>>> several >>>>>>>>>>> power outages where the batteries lasted JUST long enough to keep >>>>>>>>>>> her up. >>>>>>>>>>> >>>>>>>>>>> After all of this, I find I may have to reboot after all. Because >>>>>>>>>>> there >>>>>>>>>>> is a [EMAIL PROTECTED] process running, consuming 100% CPU (note >>>>>>>>>>> the load average), >>>>>>>>>>> and I cannot seem to kill it: >>>>>>>>>>> >>>>>>>>>>> ast% ps auxw | grep modprobe >>>>>>>>>>> root 17744 99.9 0.0 2688 412 ? RN Nov03 23223:01 >>>>>>>>>>> modprobe >>>>>>>>>>> -r ipt_state >>>>>>>>>>> ast% ps ealx | grep modprobe | grep -v grep >>>>>>>>>>> 4 0 17744 1 39 19 2688 412 - RN ? 23223:38 >>>>>>>>>>> modprobe -r ipt_state >>>>>>>>>>> ast% sudo kill 17744 >>>>>>>>>>> ast% sudo kill 17744 >>>>>>>>>>> ast% sudo kill -9 17744 >>>>>>>>>>> ast% sudo kill -9 17744 >>>>>>>>>>> ast% !ps >>>>>>>>>>> ps ealx | grep modprobe | grep -v grep >>>>>>>>>>> 4 0 17744 1 39 19 2688 412 - RN ? 23224:41 >>>>>>>>>>> modprobe -r ipt_state >>>>>>>>>>> ast% >>>>>>>>>>> >>>>>>>>>>> You may also notice that I tried "renice" to bump it all the way to >>>>>>>>>>> +19 >>>>>>>>>>> and still it consumes 100% of the CPU. The result for asterisk is >>>>>>>>>>> that I >>>>>>>>>>> hear bits of robot noise during conversations, which is annoying as >>>>>>>>>>> hell >>>>>>>>>>> but not neccessarily show stopping. But for another 19 days?? >>>>>>>>>>> Argg! >>>>>>>>>>> >>>>>>>>>>> I assume that because it is 'modprobe' it has tickled some kernel >>>>>>>>>>> bug >>>>>>>>> that >>>>>>>>>>> is merrily spinning away and won't respond to interrupts. I even >>>>>>>>>>> tried >>>>>>>>> to >>>>>>>>>>> stop it with gdb and strace, both of which also hung and had to be >>>>>>>>>>> killed >>>>>>>>>>> with -9. >>>>>>>>>>> >>>>>>>>>>> It seems to be related to me screwing with the iptables a few weeks >>>>>>>>>>> ago. >>>>>>>>>>> >>>>>>>>>>> Any ideas other than rebooting? >>>>>>>>>>> >>>>>>>>>>> Cheers, >>>>>>>>>>> >>>>>>>>>>> j >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> asterisk-users mailing list >>>>>>>>>>> To UNSUBSCRIBE or update options visit: >>>>>>>>>>> http://lists.digium.com/mailman/listinfo/asterisk-users >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> asterisk-users mailing list >>>>>>>>>>> To UNSUBSCRIBE or update options visit: >>>>>>>>>>> http://lists.digium.com/mailman/listinfo/asterisk-users >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com -- >>>>>>>>>> >>>>>>>>>> asterisk-users mailing list >>>>>>>>>> To UNSUBSCRIBE or update options visit: >>>>>>>>>> http://lists.digium.com/mailman/listinfo/asterisk-users >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com -- >>>>>>>>>> >>>>>>>>>> asterisk-users mailing list >>>>>>>>>> To UNSUBSCRIBE or update options visit: >>>>>>>>>> http://lists.digium.com/mailman/listinfo/asterisk-users >>>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com -- >>>>>>>>> >>>>>>>>> asterisk-users mailing list >>>>>>>>> To UNSUBSCRIBE or update options visit: >>>>>>>>> http://lists.digium.com/mailman/listinfo/asterisk-users >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com -- >>>>>>>>> >>>>>>>>> asterisk-users mailing list >>>>>>>>> To UNSUBSCRIBE or update options visit: >>>>>>>>> http://lists.digium.com/mailman/listinfo/asterisk-users >>>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com -- >>>>>>>> >>>>>>>> asterisk-users mailing list >>>>>>>> To UNSUBSCRIBE or update options visit: >>>>>>>> http://lists.digium.com/mailman/listinfo/asterisk-users >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Thanks, >>>>>>> Steve Totaro >>>>>>> +18887771888 (Toll Free) >>>>>>> +12409381212 (Cell) >>>>>>> +12024369784 (Skype) >>>>>>> >>>>>>> _______________________________________________ >>>>>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com -- >>>>>>> >>>>>>> asterisk-users mailing list >>>>>>> To UNSUBSCRIBE or update options visit: >>>>>>> http://lists.digium.com/mailman/listinfo/asterisk-users >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com -- >>>>>> >>>>>> asterisk-users mailing list >>>>>> To UNSUBSCRIBE or update options visit: >>>>>> http://lists.digium.com/mailman/listinfo/asterisk-users >>>>>> >>>>> >>>>> _______________________________________________ >>>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com -- >>>>> >>>>> asterisk-users mailing list >>>>> To UNSUBSCRIBE or update options visit: >>>>> http://lists.digium.com/mailman/listinfo/asterisk-users >>>>> >>>> >>>> _______________________________________________ >>>> -- Bandwidth and Colocation Provided by http://www.api-digital.com -- >>>> >>>> asterisk-users mailing list >>>> To UNSUBSCRIBE or update options visit: >>>> http://lists.digium.com/mailman/listinfo/asterisk-users >>>> >>> >>> >>> >>> -- >>> Thanks, >>> Steve Totaro >>> +18887771888 (Toll Free) >>> +12409381212 (Cell) >>> +12024369784 (Skype) >>> >>> _______________________________________________ >>> -- Bandwidth and Colocation Provided by http://www.api-digital.com -- >>> >>> asterisk-users mailing list >>> To UNSUBSCRIBE or update options visit: >>> http://lists.digium.com/mailman/listinfo/asterisk-users >>> >> >> _______________________________________________ >> -- Bandwidth and Colocation Provided by http://www.api-digital.com -- >> >> asterisk-users mailing list >> To UNSUBSCRIBE or update options visit: >> http://lists.digium.com/mailman/listinfo/asterisk-users >> > > > > -- > Thanks, > Steve Totaro > +18887771888 (Toll Free) > +12409381212 (Cell) > +12024369784 (Skype) > > _______________________________________________ > -- Bandwidth and Colocation Provided by http://www.api-digital.com -- > > asterisk-users mailing list > To UNSUBSCRIBE or update options visit: > http://lists.digium.com/mailman/listinfo/asterisk-users > _______________________________________________ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-users mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-users