Hi there, I am surprised that modifying the bulk provisioning script made a difference. The two database rows that you mentioned are functionally equivalent – the `known_preferred` field is no longer used, and Clearwater will use a quality of protection of “auth if ” `digest_qop` is not filled in.
Are you able to register a SIP phone using one of the bulk provisioned numbers? If so, are you able to register two phones to different bulk provisioned numbers and make a call between them. If this works, then the problem definitely lies with the SIPp script somewhere. The next step is to reduce the number of virtual subscribers that the SIPp nodes create, and get some logs from the running system. To do this: - Turn on debug logging on your sprout node. By default the logging level is set to log level 2, which only includes errors and very high level events. To enable debug logs, change the log level to 5 by writing log_level=5 to /etc/clearwater/user_settings (creating it if it doesn't exist already), and restart sprout (using service sprout stop - monit automatically restarts it). Sprout will now be writing debug logs to /var/log/sprout/sprout_*. - Edit /usr/share/clearwater/infrastructure/scripts/sip-stress, and set the `count` variable at the top of the file to `2`. - Restart SIP stress by running `sudo service clearwater-sip-stress stop; sudo service clearwater-infrastructure restart; sudo service clearwater-sip-stress start` Could you try this out and share the output from the most recent logs from /var/log/sprout and /var/log/clearwater-sip-stress Thanks, Alex. From: Doctor Mescaline [mailto:[email protected]] Sent: 05 November 2014 15:46 To: Alex Hockey Cc: [email protected] Subject: Re: [Clearwater] Benchmarking tool Hi Alex, every node works well with ‘sudo monit status’. The Clearwater Live Tests pass, maybe because they perform “sigh up” with ellis to get authorization. From my understanding, what the “provision subscriber” does is to directly inject the authorization of the numbers into the database, hence skipping the sign up step on ellis. If I run the “provision subscriber” I get this entry private_id | digest_ha1 | digest_qop | digest_realm | known_preferred ----------------------------+----------------------------------+------------+-----------------+----------------- [email protected]<mailto:[email protected]> | bcdbbf195231a7c2285b713b373aa70a | null | demo.clearwater | null So I modified the live test suite to register with existing numbers and passwords in such records instead of signing up with ellis. But it didn’t work util I notice that the record for an authorized number should be like: private_id | digest_ha1 | digest_qop | digest_realm | known_preferred ----------------------------+----------------------------------+------------+-----------------+----------------- [email protected]<mailto:[email protected]> | bcdbbf195231a7c2285b713b373aa70a | auth | demo.clearwater | True Then I modified the provision script to form up the records in above way and this time the test live suite works well with basic calls, which means that such numbers are available for register-make a call-unregister. But under such configuration we still get the stress test node failed (0 success call in the output csv file) with messages like: 2014-11-04 07:15:38:733 1415081738.733320: Call-Id: [email protected]<mailto:[email protected]>, receive timeout on message Call Load Test:19 without label to jump to (ontimeout attribute): aborting call. The logs of other nodes seem normal except the bono node showing some 503. Thanks Il giorno 05/nov/2014, alle ore 12:59, Alex Hockey <[email protected]<mailto:[email protected]>> ha scritto: Hi there, Provisioning the subscribers after turning up the SIPp node should work. You are seeing two types of “disconnected” logs from Bono: - The 401s are expected. The first time a client registers with Clearwater it will not provide authentication credentials. Sprout rejects the REGISTER message with a 401 response, containing an authentication challenge. The client then sends another REGISTER with credentials. - The 503s are not expected. I suspect these are responses to the 2nd, authenticated REGISTER message. They suggest a part of your deployment is not running properly. The first step is to check that all the Clearwater processes are running successfully. Run `sudo monit status` on a node to check this, and look out for any entries that are reported as failed or non-existent. If you are using an all-in-one node, just run `sudo monit status` from that node. If you are running a multi-node deployment, run the command on all your nodes. Alex. From: Doctor Mescaline [mailto:[email protected]] Sent: 04 November 2014 12:17 To: Alex Hockey Cc: [email protected]<mailto:[email protected]> Subject: Re: [Clearwater] Benchmarking tool Hi Alex, I already did it and calls were always disconnected with REASON=401 or REASON=503. I created the new SIPp node and then bulked provision subscribers, while you suggest to bulk provision subscribers and then create the new SIPp node. Is this order important? Thanks Il giorno 04/nov/2014, alle ore 12:47, Alex Hockey <[email protected]<mailto:[email protected]>> ha scritto: Hi there, I suspect the problem here is that you haven’t provisioned any subscribers on your deployment. When the SIPp nodes try to register, clearwater rejects their requests with a 403 Forbidden. The stress testing wiki page (https://github.com/Metaswitch/clearwater-docs/wiki/Clearwater-stress-testing) was missing a “provision subscribers” step. I have updated it so that is contains a link to the instructions for how to do this. It is a fairly simple process, that just involves running some scripts on your homer and homestead nodes. I hope this helps. We haven’t had many people running stress against a manually installed system before, and you’ve helped us find some gaps in our documentation! Thanks, Alex. From: The Doctor [mailto:[email protected]] Sent: 31 October 2014 15:58 To: Alex Hockey Subject: Re: [Clearwater] Benchmarking tool Hi Alex, I got it. I have configured the file and installed the package, but it 'is not able to successfully run the scenario. When running the stress test, I got these message in the bono node UTC Call-Disconnected: CALL_ID=7000069028///[email protected]<mailto:[email protected]> REASON=403 Do you think the CALL_ID format is correct? Thanks 2014-10-31 14:03 GMT+01:00 Alex Hockey <[email protected]<mailto:[email protected]>>: Hi there, I assume that you’ve set up a Clearwater deployment using the Manual Install instructions (https://github.com/Metaswitch/clearwater-docs/wiki/Manual-Install). The SIP stress node should be a completely new virtual machine. You should bootstrap this machine in the same way you bootstrapped the nodes as part of setting up the deployment (this bit wasn’t clear in the page I pointed you at). You can then install clearwater-sip-stress as a normal debian package by running `sudo apt-get install clearwater-sip-stress`. Alex. p.s. Please could you CC the list on responses? That way other members of the community benefit from the discussion :-). From: The Doctor [mailto:[email protected]<mailto:[email protected]>] Sent: 31 October 2014 11:02 To: Alex Hockey Subject: Re: [Clearwater] Benchmarking tool Hi Alex, I have read the wiki, but I don't understand where I can downaload the clearwater-sip-stress Debian package and in which node I have to edit the /etc/clearwater/config. Thanks 2014-10-31 11:07 GMT+01:00 Alex Hockey <[email protected]<mailto:[email protected]>>: Hi there, In the Project Clearwater team we use our clearwater-sip-stress package for stress testing and benchmarking. This is built on SIPp (http://sipp.sourceforge.net/). There is more information about clearwater-sip-stress on the wiki (https://github.com/Metaswitch/clearwater-docs/wiki/Clearwater-stress-testing). By default clearwater-sip-stress runs a fairly simple scenario, where simulated endpoints REGISTER, then periodically re-REGISTER and make simple calls. However you can edit the SIPp scenario file to emulate different behaviour. I'm not sure if this is what you're after, but I hope it helps. Other users on the list may have better suggestions though! Alex. -----Original Message----- From: [email protected]<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] On Behalf Of The Doctor Sent: 31 October 2014 09:12 To: [email protected]<mailto:[email protected]> Subject: [Clearwater] Benchmarking tool Hi list, I am looking for a benchmarking tool in order to emulates users behavior according to variours workload patterns. Is there any tool or Step-by-Step guide to do that? Thanks _______________________________________________ Clearwater mailing list [email protected]<mailto:[email protected]> http://lists.projectclearwater.org/listinfo/clearwater _______________________________________________ Clearwater mailing list [email protected] http://lists.projectclearwater.org/listinfo/clearwater
