Hi Linda, Did you receive Adam's email, asking for additional logs and config which we'd need to debug this?
He asked for: * The contents of /var/log/homestead/ on the Dime node, with potentially increased logging level, so we can spot errors when it reads from the Cassandra subscriber store. * The /etc/clearwater/shared_config config file. * The output of `sudo monit summary`. Also, you mentioned a 'Bind Node' - this isn't part of Project Clearwater (you can see what is part of Project Clearwater at http://www.projectclearwater.org/technical/clearwater-architecture/). Could you confirm how you created this deployment - for example, was it from OpenStack Heat templates? Thanks, Ben -----Original Message----- From: Clearwater [mailto:clearwater-boun...@lists.projectclearwater.org] On Behalf Of clearwater-requ...@lists.projectclearwater.org Sent: 06 March 2018 02:06 To: clearwater@lists.projectclearwater.org Subject: Clearwater Digest, Vol 59, Issue 12 Send Clearwater mailing list submissions to clearwater@lists.projectclearwater.org To subscribe or unsubscribe via the World Wide Web, visit http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org or, via email, send a message with subject or body 'help' to clearwater-requ...@lists.projectclearwater.org You can reach the person managing the list at clearwater-ow...@lists.projectclearwater.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Clearwater digest..." Today's Topics: 1. ??: [Clearwater] 480 Temporarily Unavailable (wang wulin) ---------------------------------------------------------------------- Message: 1 Date: Tue, 6 Mar 2018 02:05:30 +0000 From: wang wulin <wangwu...@hotmail.com> To: "Clearwater@lists.projectclearwater.org" <Clearwater@lists.projectclearwater.org> Subject: [Project Clearwater] ??: [Clearwater] 480 Temporarily Unavailable Message-ID: <kl1pr0301mb21207021c492fa72ddb45037b9...@kl1pr0301mb2120.apcprd03.prod.outlook.com> Content-Type: text/plain; charset="gb2312" I checked the syslog on Bind Node, it says: Description: clearwater-cluster-manager is exiting due to missing configuration. @@Cause: clearwater-cluster-manager was started without the mandatory etcd_cluster_key parameter. @@Effect: Datastore cluster management services are no longer available. @@Action: Verify that the configuration file /etc/clearwater/local_config is correct according to the documentation. Anyone would give some instructions about how to config etcd_cluster_key ? Thanks, Linda ________________________________ ???: wang wulin <wangwu...@hotmail.com> ????: 2018?3?6? 9:23 ???: Clearwater@lists.projectclearwater.org ??: [Clearwater] 480 Temporarily Unavailable Hi Clearwater Team, I tried with "nice -n-20 /usr/share/clearwater/bin/sipp -i 10.67.79.16 -sf ./sip-stress.xml 10.67.79.17 -t tn -s clearwater.opnfv -inf ./users.csv.1 -users 1000 -m 1000 -default_behaviors all,-bye -max_socket 65000 -max_reconnect -1 -reconnect_sleep 0 -reconnect_close 0 -send_timeout 4000 -recv_timeout 12000" and the statistics shows here: https://hastebin.com/inekabihin.sql 2018-03-05 22:55:36:131 1520290536.131506: Aborting call on unexpected message for Call-Id '679-28944@10.67.79.16': while expecting 'INVITE' (index 23), received 'SIP/2.0 480 Temporarily Unavailable Via: SIP/2.0/TCP 10.67.79.16:8412;rport=8412;received=10.67.79.16;branch=z9hG4bK-2010000642-679-30.000000-1 Record-Route: <sip:scscf.sprout.clearwater.local:5054;transport=TCP;lr;billing-role=charge-term> Record-Route: <sip:scscf.sprout.clearwater.local:5054;transport=TCP;lr;billing-role=charge-orig> Record-Route: <sip:10.67.79.17:5058;transport=TCP;lr> Record-Route: <sip:n409qMON4B@bono-i2sn7d.clearwater.local:5060;transport=TCP;lr> Call-ID: 2010000642-30.000000///679-28944@10.67.79.16 From: <sip:2010000642@clearwater.opnfv>;tag=28944SIPpTag006791234 To: <sip:2010000643@clearwater.opnfv>;tag=z9hG4bKPj33yF8tC.WrHnJKCwT2P.NJawKhULQPY1 CSeq: 546 INVITE Content-Length: 0 Thanks, Linda ________________________________ ???: wang wulin <wangwu...@hotmail.com> ????: 2018?3?2? 16:39 ???: clearwater@lists.projectclearwater.org ??: ??: [Clearwater] Could not get subscriber data from HSS Hi Clearwater Team, Here is the result after running ""/usr/share/clearwater/bin/run_stress clearwater.opnfv 16 10 "." 2018-03-02 08:25:12.757782 1519979112.757782: Aborting call on unexpected message for Call-Id '1-21165@10.67.79.16': while expecting '183' (index 2), received 'SIP/2.0 480 Temporarily Unavailable Via: SIP/2.0/TCP 10.67.79.16:54572;received=10.67.79.16;branch=z9hG4bK-21165-1-0 Record-Route: <sip:scscf.sprout.clearwater.local:5054;transport=TCP;lr;billing-role=charge-term> Record-Route: <sip:scscf.sprout.clearwater.local:5054;transport=TCP;lr;billing-role=charge-orig> Record-Route: <sip:10.67.79.17:5058;transport=TCP;lr> Record-Route: <sip:/bDGU121V2@bono-i2sn7d.clearwater.local:5060;transport=TCP;lr> Call-ID: 1-21165@10.67.79.16 From: <sip:2770000012@clearwater.opnfv>;tag=21165SIPpTag001 To: <sip:2770000015@clearwater.opnfv>;tag=z9hG4bKPjtk6L-d0QjzkZB8rbq-CMdJGW9zpHhMbt CSeq: 1 INVITE Content-Length: 0 Total calls: 1 Successful calls: 0 (0.0%) Failed calls: 1 (100.0%) Unfinished calls: 0 Retransmissions: 0 Average time from INVITE to 180 Ringing: 0.0ms # of calls with 0-2ms from INVITE to 180 Ringing: 0 (0.0%) # of calls with 2-10ms from INVITE to 180 Ringing: 0 (0.0%) # of calls with 10-20ms from INVITE to 180 Ringing: 0 (0.0%) # of calls with 20-50ms from INVITE to 180 Ringing: 0 (0.0%) # of calls with 50-100ms from INVITE to 180 Ringing: 0 (0.0%) # of calls with 100-200ms from INVITE to 180 Ringing: 0 (0.0%) # of calls with 200-500ms from INVITE to 180 Ringing: 0 (0.0%) # of calls with 500-1000ms from INVITE to 180 Ringing: 0 (0.0%) # of calls with 1000-2000ms from INVITE to 180 Ringing: 0 (0.0%) # of calls with 2000+ms from INVITE to 180 Ringing: 0 (0.0%) Failed: call success rate 0.0% is lower than target 100.0%! Total re-REGISTERs: 5 Successful re-REGISTERs: 5 (100.0%) Failed re-REGISTERS: 0 (0.0%) REGISTER retransmissions: 0 Average time from REGISTER to 200 OK: 12.0ms Thanks, Linda ________________________________ ???: wang wulin <wangwu...@hotmail.com> ????: 2018?3?2? 15:38 ???: clearwater@lists.projectclearwater.org ??: [Clearwater] Could not get subscriber data from HSS Hi Clearwater Team, I deployed a stress node according to this guidance: http://clearwater.readthedocs.io/en/stable/Clearwater_stress_testing.html, and tried to run stress via "nice -n-20 /usr/share/clearwater/bin/sipp -i 10.67.79.16 -sf ./sip-stress.xml 10.67.79.17 -t tn -s clearwater.opnfv -inf ./users.csv.1 -users 1000 -m 1000 -default_behaviors all,-bye -max_socket 65000 -max_reconnect -1 -reconnect_sleep 0 -reconnect_close 0 -send_timeout 4000 -recv_timeout 12000 ". "Register" works now, but Call step still failed, please see below for detailed info: https://hastebin.com/inekabihin.sql 2018-03-05 22:55:36:131 1520290536.131506: Aborting call on unexpected message for Call-Id '679-28944@10.67.79.16': while expecting 'INVITE' (index 23), received 'SIP/2.0 480 Temporarily Unavailable Via: SIP/2.0/TCP 10.67.79.16:8412;rport=8412;received=10.67.79.16;branch=z9hG4bK-2010000642-679-30.000000-1 Record-Route: <sip:scscf.sprout.clearwater.local:5054;transport=TCP;lr;billing-role=charge-term> Record-Route: <sip:scscf.sprout.clearwater.local:5054;transport=TCP;lr;billing-role=charge-orig> Record-Route: <sip:10.67.79.17:5058;transport=TCP;lr> Record-Route: <sip:n409qMON4B@bono-i2sn7d.clearwater.local:5060;transport=TCP;lr> Call-ID: 2010000642-30.000000///679-28944@10.67.79.16 From: <sip:2010000642@clearwater.opnfv>;tag=28944SIPpTag006791234 To: <sip:2010000643@clearwater.opnfv>;tag=z9hG4bKPj33yF8tC.WrHnJKCwT2P.NJawKhULQPY1 CSeq: 546 INVITE Content-Length: 0 I got the error from /var/log/sprout: "Error hssconnection.cpp:704: Could not get subscriber data from HSS" I only executed the 4 commands below on Vellum Node: 1) . /etc/clearwater/config; for DN in {2770000000..2770000099} ; do echo sip:$DN@$home_domain,$d...@clearwater.opnf<mailto:d...@clearwater.opnf>v,clearwater.opnfv,7kkzTyGW ;done > users.csv 2) cd /usr/share/clearwater/crest-prov/src/metaswitch/crest/tools/ && python bulk_create.py users.csv 3) ./users.create_xdm.sh 4) ./users.create_homestead.sh Did I miss some other steps? Do you know if we "HSS" node is also required? 2) I also got the error from Dime node: root@dime-5y29tl:/var/log/ralf# vim /var/log/syslog Mar 2 06:37:01 dime-5y29tl issue-alarm: zmq_msg_recv: Invalid argument Mar 2 06:37:02 dime-5y29tl config-manager[10778]: dropped request: 'issue-alarm config-manager 8500.3' Mar 2 06:37:08 dime-5y29tl issue-alarm: zmq_msg_recv: Invalid argument Mar 2 06:37:18 dime-5y29tl issue-alarm: message repeated 12 times: [ zmq_msg_recv: Invalid argument] Mar 2 06:37:18 dime-5y29tl queue-manager[10648]: dropped request: 'issue-alarm queue-manager 9001.1' Mar 2 06:37:21 dime-5y29tl queue-manager[10648]: dropped request: 'issue-alarm queue-manager 9002.1' Mar 2 06:37:22 dime-5y29tl issue-alarm: zmq_msg_recv: Invalid argument root@dime-5y29tl:/var/log/ralf# vim /var/log/monit.log [UTC Mar 2 02:31:43] error : 'poll_etcd_cluster' '/usr/share/clearwater/bin/poll_etcd_cluster.sh' failed with exit status (1) -- 1 [UTC Mar 2 02:31:43] info : 'poll_etcd_cluster' exec: /bin/bash [UTC Mar 2 02:31:53] info : 'poll_etcd_cluster' status succeeded [status=0] -- zmq_msg_recv: Resource temporarily unavailable Any help would be much appreciated! Thanks, Linda -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.projectclearwater.org/pipermail/clearwater_lists.projectclearwater.org/attachments/20180306/77191103/attachment.html> ------------------------------ Subject: Digest Footer _______________________________________________ Clearwater mailing list Clearwater@lists.projectclearwater.org http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org ------------------------------ End of Clearwater Digest, Vol 59, Issue 12 ****************************************** _______________________________________________ Clearwater mailing list Clearwater@lists.projectclearwater.org http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org
