Hi Shree, Yes, http://clearwater.readthedocs.org/en/latest/Clearwater_stress_testing/index.html is the right set of instructions. The clearwater-sip-stress package is available from our Debian package server at http://repo.cw-ngv.com/stable - just follow the “Configure the APT software sources” section of our manual install guide (http://clearwater.readthedocs.org/en/latest/Manual_Install/index.html#configure-the-apt-software-sources) to add that repo server, and then run ‘sudo apt-get update && sudo apt-get install clearwater-sip-stress’.
You might also want to read our blog post on Clearwater performance (http://www.projectclearwater.org/clearwater-performance-and-our-load-monitor/) which gives some tips on stress testing – in particular, how to not accidentally trigger our overload controls. Best, Rob -- Rob Day Software Engineer, Project Clearwater From: SHREE DUTH Awasthi [mailto:[email protected]] Sent: 24 June 2015 09:46 To: Robert Day Cc: [email protected]; Ravi Chaudhary; Eleanor Merry Subject: Re: [Clearwater] Multi VM setup -SIP error 408 Hi Rob, We wanted to check the scale out capability of clearwater by increasing the load, we followed a link ( http://clearwater.readthedocs.org/en/latest/Clearwater_stress_testing/index.html ) which specified clearwater-sip-stress package, where can we find this package and is this the rite way to go ahead with. Please help us with some pointers. Thanks, Shree Duth Awasthi On Mon, Jun 22, 2015 at 11:32 PM, Robert Day <[email protected]<mailto:[email protected]>> wrote: Hi Shree, Glad to hear it’s working now! A couple of your emails were in the mailing list queue and all came through together at the same time – are you still having issues in any environments, or did fixing the cluster_settings file resolve everything? Thanks, Rob -- Rob Day Software Engineer, Project Clearwater From: [email protected]<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] On Behalf Of SHREE DUTH Awasthi Sent: 19 June 2015 14:53 To: Eleanor Merry Cc: [email protected]<mailto:[email protected]>; Ravi Chaudhary Subject: Re: [Clearwater] Multi VM setup -SIP error 408 Hello Elle, Thanks a lot Elle :) It worked after we change the cluster settings. Thanks and Regards, Shree Duth Awasthi. On Fri, Jun 19, 2015 at 4:20 PM, Eleanor Merry <[email protected]<mailto:[email protected]>> wrote: Hi, Looking in the Bono logs, the WWW-Authenticate header in the 401 response has the parameter stale=True: WWW-Authenticate: Digest realm="11.0.0.7",nonce="078b9e1a1d596ecf",opaque="149b89ad75606ac6",stale=true,algorithm=MD5,qop="auth" Sprout can set this if it can’t find the authentication credentials in Memcached for the subscriber (which it uses to check against the REGISTER). In this case, it queries Homestead again for the authentication vector, tries to store the credentials, and sends back a 401 with stale=True. However, because you’re seeing this error repeatedly, this suggests to me that there are some memcached errors on your Sprout node. Can you please confirm that memcached is running on Sprout, and send me the Sprout logs? Ellie From: SHREE DUTH Awasthi [mailto:[email protected]<mailto:[email protected]>] Sent: 18 June 2015 20:52 To: Eleanor Merry; Ravi Chaudhary Cc: [email protected]<mailto:[email protected]> Subject: Re: [Clearwater] Multi VM setup -SIP error 408 Hello Elle / All, Thanks for your help so far. We were able to succeed with the SIP call after your suggestion. The same images we have deployed on openstack environment and associated floating IPs and all of the services are up and running. Here again we are unable to make a SIP call using "Zoiper" may be due to the attached continuous error logs from Bono. Any help is much appreciated. PFA. Thanks in advance. Best Regards, Shree Duth Awasthi. On Thu, Apr 9, 2015 at 10:33 PM, Eleanor Merry <[email protected]<mailto:[email protected]>> wrote: Hi Shree You’ll need to change your Zoiper configuration to use your home_domain value in /etc/clearwater/config as the domain (so use 10.90.10.104 rather than example.com<http://example.com>). Ellie One other thing - I've noticed you're not signed up to the mailing list. This means your mails to the list do not automatically get posted. Would you be able to sign up please? You can do this by following the link from http://www.projectclearwater.org/community/. From: SHREE DUTH Awasthi [mailto:[email protected]<mailto:[email protected]>] Sent: 09 April 2015 07:28 To: Eleanor Merry; Ravi Chaudhary Cc: [email protected]<mailto:[email protected]> Subject: Re: [Clearwater] Multi VM setup -SIP error 408 Hi Elle / All, A small updation below. (Forgot to mention random password in the mail) Other values in Zoiper: Account Type : SIP user / user@host : 2010000187 (As displayed in Ellis UI) password:cVf5DpkTq (As displayed in Ellis UI) Domain / Outbound Proxy : example.com<http://example.com/> Account name: [email protected]<mailto:[email protected]> Skip Auto detection : enabled Username: 2010000187 Password: cVf5DpkTq Caller ID Name: [email protected]<mailto:[email protected]> Outbound Proxy: 10.90.10.104:5060<http://10.90.10.104:5060/> (<Bono-IP>:5060) Note: Same / similar values worked for us in all-in-one clearwater deployment. Thanks and Regards, Shree Duth Awasthi. On Thu, Apr 9, 2015 at 1:00 AM, SHREE DUTH Awasthi <[email protected]<mailto:[email protected]>> wrote: Hi Elle, Thanks a lot for your time and response. We are using the Bono IP 10.90.10.104 as home_domain in file /etc/clearwater/config in all the VMs on ESX Hypervisor. when registering an account using Zoiper / Xlite, the domain is set as "example.com<http://example.com>" Please let us know, if this is incorrect. We sometimes get SIP error 408 (request timeout) or 403 (Forbidden). Other values in Zoiper: Account Type : SIP user / user@host : 2010000187 (As displayed in Ellis UI) password:cVf5DpkTq (As displayed in Ellis UI) Domain / Outbound Proxy : example.com<http://example.com> Account name: [email protected]<mailto:[email protected]> Skip Auto detection : enabled Username: 2010000187 Password: Caller ID Name: [email protected]<mailto:[email protected]> Outbound Proxy: 10.90.10.104:5060<http://10.90.10.104:5060> (<Bono-IP>:5060) Interesting logs below. Complete sprout and chronous logs attached. Homer: Traceback (most recent call last): Failure: twisted.internet.error.ConnectionRefusedError: Connection was r efused by other side: 111: Connection refused. Bono: 08-04-2015 18:12:38.813 UTC Call-Disconnected: CALL_ID=M2E1MGJlZTMyM2Y4ODJkOTBlMjhmODIwMDFiYTgzNGE. REASON=403 Sprout logs: SIP/2.0 200 OK Via: SIP/2.0/TCP 10.90.10.235;rport=39631;received=10.90.10.235;branch=z9 hG4bK-27391 Call-ID: poll-sip-27391 From: "poll-sip" <sip:[email protected]<mailto:sip%[email protected]>>;tag=27391 To: <sip:[email protected]<mailto:sip%[email protected]>>;tag=z9hG4bK-27391 CSeq: 27391 OPTIONS Content-Length: 0 --end msg-- 08-04-2015 19:05:58.221 UTC Debug pjsip: tdta0x7f58382b Destroying txdata Response msg 200/OPTIONS/cseq=27391 (tdta0x7f58382bb110) 08-04-2015 19:05:58.221 UTC Debug thread_dispatcher.cpp:176: Worker threa d completed processing message 0x7f57f8013b18 08-04-2015 19:05:58.221 UTC Debug thread_dispatcher.cpp:182: Request late ncy = 325us 08-04-2015 19:05:58.231 UTC Verbose httpstack.cpp:262: Process request fo r URL /ping, args (null) 08-04-2015 19:05:58.231 UTC Verbose httpstack.cpp:69: Sending response 20 0 to request for URL /ping, args (null) 08-04-2015 19:05:59.227 UTC Verbose pjsip: tcps0x7f57f80c TCP connection closed 08-04-2015 19:05:59.228 UTC Debug connection_tracker.cpp:91: Connection 0x7f57f80cab08 has been destroyed 08-04-2015 19:05:59.228 UTC Verbose pjsip: tcps0x7f57f80c TCP transport destroyed with reason 70016: End of file (PJ_EEOF) Ralf logs: rt=UDP", "Time-Stamps": { "SIP-Request-Timestamp": 1428520095, "SIP-Request-Timestamp-Fraction": 447, "SIP-Response-Timestamp": 1428520095, "SIP-Response-Timestamp-Fraction": 459 }, "IMS-Charging-Identifier": "", "Cause-Code": 403, "From-Address": "\"2010000187\" <sip:2010000187@exa mple.com<http://mple.com>>;tag=684dbf26", "IMS-Visited-Network-Identifier": "10.90.10.104", "Route-Header-Received": "<sip:10.90.10.235:5054;tr ansport=TCP;lr;orig<sip:10.90.10.235:5054;tr%20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 ansport=TCP;lr;orig>>" } } } } 08-04-2015 19:08:15.464 UTC Error handlers.cpp:222: JSON lacked a ' ccf' array, or the array was empty (mandatory for START/EVENT) 08-04-2015 19:08:15.464 UTC Verbose httpstack.cpp:68: Sending respo nse 400 to request for URL /call-id/NmQ5ZDQ1MGI5MTNkZWVjYTk5ZDc1YjI 0YjQ2N2QwMzk., args {"peers":{},"event":{"Accounting-Record-Type":1 ,"Event-Timestamp":1428520095,"Service-Information":{"Subscription- Id":[{"Subscription-Id-Type":2,"Subscription-Id-Data":"sip:20100001 [email protected]<mailto:[email protected]>"}],"IMS-Information":{"Event-Type":{"SIP-Method":"RE GISTER","Expires":3600},"Role-Of-Node":0,"Node-Functionality":0,"Us er-Session-Id":"NmQ5ZDQ1MGI5MTNkZWVjYTk5ZDc1YjI0YjQ2N2QwMzk.","Call ed-Party-Address":"sip:[email protected]<mailto:sip%[email protected]>","Requested-Party-Add ress":"sip:example.com<http://example.com>;transport=UDP","Time-Stamps":{"SIP-Request-T imestamp":1428520095,"SIP-Request-Timestamp-Fraction":447,"SIP-Resp onse-Timestamp":1428520095,"SIP-Response-Timestamp-Fraction":459}," IMS-Charging-Identifier":"","Cause-Code":403,"From-Address":"\"2010 000187\" <sip:[email protected]<mailto:sip%[email protected]>>;tag=684dbf26","IMS-Visited-Ne twork-Identifier":"10.90.10.104","Route-Header-Received":"<sip:10.9 0.10.235:5054;transport=TCP;lr;orig<sip:10.9%20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 %20 0.10.235:5054;transport=TCP;lr;orig>>"}}}} 08-04-2015 19:08:19.619 UTC Verbose httpstack.cpp:259: Process request for URL /ping, args (null) 08-04-2 Thanks and Regards, Shree Duth Awasthi. +91 8867713871<tel:%2B91%208867713871> On Thu, Apr 9, 2015 at 12:02 AM, Eleanor Merry <[email protected]<mailto:[email protected]>> wrote: Hi, There weren't any INVITE attempts in the attached Sprout logs. However, looking at the Zoiper screenshot, it looks like the Zoiper client isn't registering (and so won't be able to be part of a call). In Zoiper, you should set the domain to be the same value as the home_domain value in /etc/clearwater/config. Your auth username should be the private identity of your subscriber (which I'd expect to be <username>@<domain>). If this doesn't work, please can you send me the debug logs from Sprout for a REGISTER attempt? Ellie ________________________________ From: SHREE DUTH Awasthi [[email protected]<mailto:[email protected]>] Sent: 02 April 2015 22:23 To: Eleanor Merry Cc: [email protected]<mailto:[email protected]> Subject: Re: [Clearwater] Multi VM setup -SIP error 408 Hi Elle, Thanks a lot for your time and suggestions. Yes, we have created the file "/etc/clearwater/user_settings" as suggested and stopped the sprout service. Yes, we may have to ignore the CDF errors as of now. .PFA for the setting used for calling and requested logs. Please suggest if any changes are required. Best Regards, Shree Duth Awasthi. Noticable Sprout logs: SIP/2.0 200 OK Via: SIP/2.0/TCP 127.0.0.1;rport=45687;received=127.0.0.1;branch=z9hG4bK-34765 Call-ID: poll-sip-34765 From: "poll-sip" <sip:[email protected]<mailto:sip%[email protected]>>;tag=34765 To: <sip:[email protected]<mailto:sip%[email protected]>>;tag=z9hG4bK-34765 CSeq: 34765 OPTIONS Content-Length: 0 --end msg-- 02-04-2015 21:20:33.429 UTC Debug pjsip: tdta0x7f10dc00 Destroying txdata Response msg 200/OPTIONS/cseq=34765 (tdta0x7f10dc00ee40) 02-04-2015 21:20:33.429 UTC Debug thread_dispatcher.cpp:176: Worker thread completed processing message 0x7f10d400c4a8 02-04-2015 21:20:33.429 UTC Debug thread_dispatcher.cpp:182: Request latency = 494us 02-04-2015 21:20:33.430 UTC Debug statistic.cpp:103: Send new value for statistic latency_us, size 5 02-04-2015 21:20:33.430 UTC Debug zmq_lvc.cpp:172: Update to latency_us statistic 02-04-2015 21:20:33.430 UTC Debug zmq_lvc.cpp:256: Clearing message cache for 0x7f110c012ff0 02-04-2015 21:20:33.440 UTC Verbose httpstack.cpp:262: Process request for URL /ping, args (null) 02-04-2015 21:20:33.440 UTC Verbose httpstack.cpp:69: Sending response 200 to request for URL /ping, args (null) 02-04-2015 21:20:34.438 UTC Verbose pjsip: tcps0x7f10d400 TCP connection closed 02-04-2015 21:20:34.438 UTC Debug connection_tracker.cpp:91: Connection 0x7f10d4007f38 has been destroyed 02-04-2015 21:20:34.438 UTC Verbose pjsip: tcps0x7f10d400 TCP transport destroyed with reason 70016: End of file (PJ_EEOF) ^C On Thu, Apr 2, 2015 at 9:44 AM, Eleanor Merry <[email protected]<mailto:[email protected]>> wrote: Hi Shree, Can you please turn on debug logging for Sprout, retry the call, and then send me the debug logs? You can turn on debug logging by creating/editing the file '/etc/clearwater/user_settings' on the Sprout node, add log_level=5, and then restart Sprout (run 'service sprout stop' and allow Sprout to be restarted by monit). In the logs you sent before, there are some errors where Sprout is attempting to send an ACR to Ralf and this is failing. This can happen if Ralf is unable to send the ACR on to a CDF. If you don't have a CDF (not provided by Project Clearwater) in your system then you don't need to have a Ralf in your Clearwater deployment. There are also errors where Sprout can't send timer requests to Chronos. Can you also send me the Chronos logs (in /var/log/chronos/)? Ellie -----Original Message----- From: [email protected]<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] On Behalf Of SHREE DUTH Awasthi Sent: 01 April 2015 21:44 To: [email protected]<mailto:[email protected]> Subject: Re: [Clearwater] Multi VM setup -SIP error 408 Hi All, Request your time and suggestions. Another important observation on Sprout VM: [sprout]sprout1@sprout1:/var/log/sprout$ sudo netstat -anp | grep 7253 tcp 0 0 127.0.0.1:7253<http://127.0.0.1:7253> 0.0.0.0:* LISTEN 1284/chronos tcp 0 0 255.255.255.255:7253<http://255.255.255.255:7253> 0.0.0.0:* LISTEN 1284/chronos tcp 0 0 127.0.0.1:37680<http://127.0.0.1:37680> 127.0.0.1:7253<http://127.0.0.1:7253> TIME_WAIT - tcp 0 0 127.0.0.1:37819<http://127.0.0.1:37819> 127.0.0.1:7253<http://127.0.0.1:7253> TIME_WAIT - tcp 0 0 127.0.0.1:37271<http://127.0.0.1:37271> 127.0.0.1:7253<http://127.0.0.1:7253> TIME_WAIT - tcp 0 0 127.0.0.1:37548<http://127.0.0.1:37548> 127.0.0.1:7253<http://127.0.0.1:7253> TIME_WAIT - tcp 0 0 127.0.0.1:37137<http://127.0.0.1:37137> 127.0.0.1:7253<http://127.0.0.1:7253> TIME_WAIT - tcp 0 0 127.0.0.1:37406<http://127.0.0.1:37406> 127.0.0.1:7253<http://127.0.0.1:7253> TIME_WAIT - [sprout]sprout1@sprout1:/var/log/sprout$ [sprout]sprout1@sprout1:/var/log/sprout$ nmap -sT -O localhost TCP/IP fingerprinting (for OS scan) requires root privileges. QUITTING! [sprout]sprout1@sprout1:/var/log/sprout$ sudo nmap -sT -O localhost Starting Nmap 5.21 ( http://nmap.org ) at 2015-04-01 20:39 UTC Nmap scan report for localhost (127.0.0.1) Host is up (0.000044s latency). Not shown: 998 closed ports PORT STATE SERVICE 22/tcp open ssh 53/tcp open domain No exact OS matches for host (If you know what OS is running on it, see http://nmap.org/submit/ ). TCP/IP fingerprint: OS:SCAN(V=5.21%D=4/1%OT=22%CT=1%CU=37313%PV=N%DS=0%DC=L%G=Y%TM=551C5782%P=x OS:86_64-unknown-linux-gnu)SEQ(SP=106%GCD=1%ISR=10A%TI=Z%CI=I%II=I%TS=8)SEQ OS:(SP=106%GCD=2%ISR=10A%TI=Z%CI=I%II=I%TS=8)OPS(O1=MFFD7ST11NW9%O2=MFFD7ST OS:11NW9%O3=MFFD7NNT11NW9%O4=MFFD7ST11NW9%O5=MFFD7ST11NW9%O6=MFFD7ST11)WIN( OS:W1=AAAA%W2=AAAA%W3=AAAA%W4=AAAA%W5=AAAA%W6=AAAA)ECN(R=Y%DF=Y%T=40%W=AAAA OS:%O=MFFD7NNSNW9%CC=Y%Q=)T1(R=Y%DF=Y%T=40%S=O%A=S+%F=AS%RD=0%Q=)T2(R=N)T3( OS:R=N)T4(R=Y%DF=Y%T=40%W=0%S=A%A=Z%F=R%O=%RD=0%Q=)T5(R=Y%DF=Y%T=40%W=0%S=Z OS:%A=S+%F=AR%O=%RD=0%Q=)T6(R=Y%DF=Y%T=40%W=0%S=A%A=Z%F=R%O=%RD=0%Q=)T7(R=Y OS:%DF=Y%T=40%W=0%S=Z%A=S+%F=AR%O=%RD=0%Q=)U1(R=Y%DF=N%T=40%IPL=164%UN=0%RI OS:PL=G%RID=G%RIPCK=G%RUCK=G%RUD=G)IE(R=Y%DFI=N%T=40%CD=S) Network Distance: 0 hops OS detection performed. Please report any incorrect results at http://nmap.org/submit/ . Nmap done: 1 IP address (1 host up) scanned in 11.93 seconds Br, Shree On Wed, Apr 1, 2015 at 1:26 PM, SHREE DUTH Awasthi < [email protected]<mailto:[email protected]>> wrote: > Hello All, > > Am sorry for large attachments. > > We have successfully installed clearwater multi VM setup with all > services running in six different VMs which are on ESX hyper visor > which inturn is on HP proliant BL Gen8 server. > > We tried using Zoiper / X-lite client to test the SIP call but the > call is not going through. Can you please help us with pointers ? > > Notifiable Sprout logs: > 01-04-2015 18:43:44.370 UTC Error httpconnection.cpp:624: > http://10.90.10.235:7253/timers failed at server 10.90.10.235 : > Couldn't connect to server (7) : fatal > 01-04-2015 18:43:44.370 UTC Error httpconnection.cpp:624: > http://10.90.10.235:7253/timers failed at server 10.90.10.235 : > Couldn't connect to server (7) : fatal > 01-04-2015 18:43:44.371 UTC Error httpconnection.cpp:738: cURL failure > with cURL error code 7 (see man 3 libcurl-errors) and HTTP error code > 404 > 01-04-2015 18:43:44.377 UTC Error httpconnection.cpp:738: cURL failure > with cURL error code 0 (see man 3 libcurl-errors) and HTTP error code > 400 > 01-04-2015 18:43:44.377 UTC Warning acr.cpp:627: Failed to send Ralf > ACR message (0x7f8594007480), rc = 400 > > Please find all the relevant attachemnts. > > Note: We have already tried calling with and without outboud proxy ! > > Best Regards, > Shree Duth Awasthi. > +91 8867713871<tel:%2B91%208867713871> > _______________________________________________ Clearwater mailing list [email protected]<mailto:[email protected]> http://lists.projectclearwater.org/listinfo/clearwater
_______________________________________________ Clearwater mailing list [email protected] http://lists.projectclearwater.org/listinfo/clearwater
