kind of late to diagnose. very bothersome to troubleshoot 4.2.anything. you are asking something that you already have an answer to:
http://track.sipfoundry.org/browse/XX-8063 On Thu, Sep 8, 2011 at 10:08 AM, Jan Fricke <[email protected]> wrote: > Hi, > > after a reboot everything works fine again. The problem is more complex > than it seemed on the first look. Let me explain it: > > > > The hole telephony was down. The system ran for several month without any > problem. It was reachable by ssh and http. > > sipxproc on console showed all services running. SipXconfig showed all > services undefined. About one hour before the monitoring alerted the CPU > load there were some strange lines in the proxy log: > > > > "2011-09-07T18:26:49.312936Z":3516699:KERNEL:ERR:master.voip.asdf.local:SipClientTcp-26:FFFFFFFF:SipXProxy:"OsMsgQShared::doSendCore > message send failed for queue 'SipTcpServer-3' - no room, ret = 9" > > > > Later also : > > "2011-09-08T00:20:28.966534Z":3528586:SIP:CRIT:master.voip.asdf.local:SipClientUdp-8:B7A8FB90:SipXProxy:"SipUserAgent::queueMessageToInterestedObservers > send failed with status 12 (numMsgs = 2000, maxMsgs = 2000)" > > "2011-09-08T00:20:28.966559Z":3528587:SIP:CRIT:master.voip.asdf.local:SipClientUdp-8:B7A8FB90:SipXProxy:"SipUserAgent::queueMessageToInterestedObservers > send failed to queue named 'SipRouter-11'" > > "2011-09-08T00:20:28.966577Z":3528588:SIP:CRIT:master.voip.asdf.local:SipClientUdp-8:B7A8FB90:SipXProxy:"SipUserAgent::queueMessageToInterestedObservers > observerQueue 0x9860ec4, observerData (nil), SIP method '', wantsRequests 1, > wantsResponses 0, wantsIncoming 1, wantsOutGoing 0, eventName '', SipSession > (nil)" > > > > > > When I tried to restart the presence service In the sipxpresence.log > > "2011-09-08T07:00:09.394617Z":4:KERNEL:ERR:master.voip.asdf.local:SipServerBroker-4:FFFFFFFF:sipxpresence:"OsServerSocket: > accept(3) error: 22=Invalid argument" > > "2011-09-08T07:00:29.390130Z":5:ACD:CRIT:master.voip.asdf.local:pid-3269:B7F446E0:sipxpresence:"sigHandler: > caught signal: 6" > > > > > > > > It seems that the network stack worked partially (it accepted packets on > existing socket. That’s why ssh and http worked.). But after receiving the > packets it could not process it internally. Loopback interface for internal > communication crashed? An local application that normally subscribes > presence from the rls got no answer on its subscribe request. Kernel > crashed? To me this seems like a very strange operating system issue. What > do you think? > > > > Sincerely > > > > *Von:* [email protected] [mailto: > [email protected]] *Im Auftrag von *Tony Graziano > *Gese**ndet:* Donnerstag, 8. September 2011 10:41 > *An:* Discussion list for users of sipXecs software > *Betreff:* Re: [sipx-users] SipXrls stopped working > > > > I think this has been discussed before. RLS has a performance issue in > larger installations especially with XMPP roles enabled. In your case > updating to 4.4 would help to eliminate a lot of the RLS issues, but in the > meantime have you attempted a reboot? > > On Thu, Sep 8, 2011 at 4:27 AM, Michael Picher <[email protected]> wrote: > > Jan, > > > > I'd move to 4.4.0 to pick up large improvements to RLS. > > > > Thanks, > > Mike > > On Thu, Sep 8, 2011 at 2:56 AM, Jan Fricke <[email protected]> wrote: > > Hi, > > since yesterday we have a problem with the resourcelistserver of a sipx > 4.2.1 installation. Our system monitoring threw an alert CPU 100% Load > 70. > > /var/log/messages.1 contains the following information several times: > > > > Sep 7 20:23:47 master kernel: INFO: task sipxrls:3896 blocked for more > than 120 seconds. > > Sep 7 20:23:47 master kernel: "echo 0 > > /proc/sys/kernel/hung_task_timeout_secs" disables this message. > > Sep 7 20:23:47 master kernel: sipxrls D 000C5F27 2604 3896 > 3160 3949 (NOTLB) > > Sep 7 20:23:47 master kernel: f68cde58 00000082 634ecdc0 000c5f27 > 000c5f27 f62eb000 00000092 0000000a > > Sep 7 20:23:47 master kernel: f6227550 634ecdc0 000c5f27 00000000 > 00000001 f622765c c2013944 f74c1e40 > > Sep 7 20:23:47 master kernel: 00000002 c2013944 00000000 00000001 > 00000070 00000080 018cc900 634ecdc0 > > Sep 7 20:23:47 master kernel: Call Trace: > > Sep 7 20:23:47 master kernel: [<c061c8ab>] wait_for_completion+0x6b/0x8f > > Sep 7 20:23:47 master kernel: [<c041f7a7>] default_wake_function+0x0/0xc > > Sep 7 20:23:47 master kernel: [<c0426be4>] exit_mm+0x69/0xda > > Sep 7 20:23:47 master kernel: [<c04280cd>] do_exit+0x20c/0x794 > > Sep 7 20:23:47 master kernel: [<c04286cb>] sys_exit_group+0x0/0xd > > Sep 7 20:23:47 master kernel: [<c042fddf>] > get_signal_to_deliver+0x3a2/0x3c9 > > Sep 7 20:23:48 master kernel: [<c0404583>] do_notify_resume+0x77/0x67d > > Sep 7 20:23:48 master kernel: [<c041f7a7>] default_wake_function+0x0/0xc > > Sep 7 20:23:48 master kernel: [<c044be1e>] > audit_syscall_entry+0x15a/0x18c > > Sep 7 20:23:48 master kernel: [<c044c1b8>] audit_syscall_exit+0x2da/0x301 > > Sep 7 20:23:48 master kernel: [<c0404fa6>] work_notifysig+0x13/0x19 > > Sep 7 20:23:48 master kernel: ======================= > > Sep 7 20:23:48 master kernel: INFO: task sipxrls:3949 blocked for more > than 120 seconds. > > Sep 7 20:23:48 master kernel: "echo 0 > > /proc/sys/kernel/hung_task_timeout_secs" disables this message. > > Sep 7 20:23:48 master kernel: sipxrls D 000C5F27 2868 3949 > 3160 3951 3896 (NOTLB) > > Sep 7 20:23:48 master kernel: f6295e58 00000082 635e1000 000c5f27 > 000c5f27 f62eb000 00000092 00000007 > > Sep 7 20:23:48 master kernel: f6294aa0 635e1000 000c5f27 00000000 > 00000001 f6294bac c2013944 f74c1e40 > > Sep 7 20:23:48 master kernel: 00000002 c2013944 00000000 00000000 > 00000000 c0405a89 f74c1e78 635e1000 > > Sep 7 20:23:48 master kernel: Call Trace: > > Sep 7 20:23:48 master kernel: [<c0405a89>] error_code+0x39/0x40 > > Sep 7 20:23:48 master kernel: [<c061c8ab>] wait_for_completion+0x6b/0x8f > > Sep 7 20:23:48 master kernel: [<c041f7a7>] default_wake_function+0x0/0xc > > Sep 7 20:23:48 master kernel: [<c0426be4>] exit_mm+0x69/0xda > > Sep 7 20:23:48 master kernel: [<c04280cd>] do_exit+0x20c/0x794 > > Sep 7 20:23:48 master kernel: [<c04286cb>] sys_exit_group+0x0/0xd > > Sep 7 20:23:48 master kernel: [<c042fddf>] > get_signal_to_deliver+0x3a2/0x3c9 > > Sep 7 20:23:48 master kernel: [<c0404583>] do_notify_resume+0x77/0x67d > > Sep 7 20:23:48 master kernel: [<c042f837>] dequeue_signal+0x34/0xa8 > > Sep 7 20:23:48 master kernel: [<c042e0d2>] recalc_sigpending+0xe/0x20 > > Sep 7 20:23:48 master kernel: [<c04305a1>] > sys_rt_sigtimedwait+0x253/0x2c2 > > Sep 7 20:23:48 master kernel: [<c044be1e>] > audit_syscall_entry+0x15a/0x18c > > Sep 7 20:23:48 master kernel: [<c044c1b8>] audit_syscall_exit+0x2da/0x301 > > Sep 7 20:23:48 master kernel: [<c0404fa6>] work_notifysig+0x13/0x19 > > Sep 7 20:23:48 master kernel: ======================= > > > > After a while the heavy load was gone but since then the rls does not work. > I tried to restart with > > > > sipxproc –restart ResourceListServer > > > > but it does not answer any subscribe requests an > /var/log/sipxpbx/sipxrls.log is empty. > > > > > > Any ideas? > > _____________________________ > > Jan Fricke B.Sc. Cs. > > > > IANT- APPLIED NGN-TECHNOLOGIES > > > > Schlüsselfertige VoIP-Lösungen und mehr... > > > > Member of GROUPLINK > > > > IANT GmbH > > Salzdahlumer Straße 46/48 > > D-38302 Wolfenbüttel > > Fon: +49/(0)5331/ 900989-0 > > Fax: +49/(0)5331/ 900989-499 > > > > Mail: [email protected] > > Internet: www.iant.de > > > > > > Bankverbindung: IANT GmbH, Konto-Nr. 12 95 98 50 00, Volksbank BraWo (BLZ > 269 910 66); IBAN-Nr.: DE02 2699 1066 1295 9850 00, BIC: GENODEF1WOB; > Steuer-Nr.: 51/DP2013; Ust.-IdNr: DE264352710; HRB 201710, Amtsgericht > Braunschweig; Geschäftsführer: Dipl.-Ing. Jan Schumacher, Prof. Dr.-Ing. > Diederich Wermser > > > > Diese E-Mail kann vertrauliche und/oder rechtlich geschützte Informationen > enthalten. Wenn Sie nicht der richtige Empfänger sind oder diese E-Mail > irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und > vernichten Sie diese E-Mail. > > This e-mail may contain confidential and/or privileged information. If you > are not the intended recipient or have received this e-mail in error please > notify the sender immediately and delete this e-mail. > > > > > > > > > > _______________________________________________ > sipx-users mailing list > [email protected] > List Archive: http://list.sipfoundry.org/archive/sipx-users/ > > > > > > -- > Michael Picher > eZuce > Director of Technical Services > O.978-296-1005 X2015 > M.207-956-0262 > @mpicher <http://twitter.com/mpicher> > www.ezuce.com > > > _______________________________________________ > sipx-users mailing list > [email protected] > List Archive: http://list.sipfoundry.org/archive/sipx-users/ > > > > > > -- > ====================== > Tony Graziano, Manager > Telephone: 434.984.8430 > sip: [email protected] > Fax: 434.465.6833 > > Email: [email protected] > > LAN/Telephony/Security and Control Systems Helpdesk: > Telephone: 434.984.8426 > sip: [email protected] > > Helpdesk Contract Customers: > http://support.myitdepartment.net > > > > Blog: > > http://blog.myitdepartment.net > > > > Linked-In Profile: http://www.linkedin.com/pub/tony-graziano/14/4a6/7a4 > > > > Ask about our Internet faxservices! > > > > _______________________________________________ > sipx-users mailing list > [email protected] > List Archive: http://list.sipfoundry.org/archive/sipx-users/ > -- ====================== Tony Graziano, Manager Telephone: 434.984.8430 sip: [email protected] Fax: 434.465.6833 Email: [email protected] LAN/Telephony/Security and Control Systems Helpdesk: Telephone: 434.984.8426 sip: [email protected] Helpdesk Contract Customers: http://support.myitdepartment.net <http://support.myitdepartment.net>Blog: http://blog.myitdepartment.net Linked-In Profile: http://www.linkedin.com/pub/tony-graziano/14/4a6/7a4 Ask about our Internet faxservices!
_______________________________________________ sipx-users mailing list [email protected] List Archive: http://list.sipfoundry.org/archive/sipx-users/
