Hi doc, I used the stress test with the manual installation and everything works fine. I followed the instructions on the clearwater documentation.
I may help you if you provide the installation process you performed, Ken 2014-11-18 11:52 GMT+01:00 Doctor Mescaline <[email protected]>: > Hi All, > > I am trying to use the stress testing with my manual installation, but it > doesn’t work. > > I understand that many people run stress tests against an automated > installed system and this kind of installation requires an Amazon EC2 > account. > > Is there any way to perform an automated installation without using Amazon > EC2? > > thanks > > > > > > Il giorno 08/nov/2014, alle ore 19:01, Doctor Mescaline < > [email protected]> ha scritto: > > > > Hi Alex, > > > > Thanks for your suggestions. For the provision, it made a differences, > but I do not remember if it is because I also change some other > configurations. Anyway it’s not a problem now and I will check it later. > > > > Following your instructions to reduce the counts, I also got the same > error. The situation is that I can either make real call or run live test > successfully with the preset users and password ‘7kkzTyGW’ in provision,but > I cannot get stress test passed. I suspect there’s something wrong with my > configuration of sip-stress. > > > > Below is the logs of bono, sprout and sip-stress. > > > > Thanks > > > > bono: > > > > 07-11-2014 12:02:37.671 UTC Call-Disconnected: CALL_ID=2010000000/// > [email protected] <mailto:CALL_ID=2010000000///[email protected]> > REASON=401 > > 07-11-2014 12:02:37.689 UTC Call-Disconnected: CALL_ID=2010000001/// > [email protected] <mailto:CALL_ID=2010000001///[email protected]> > REASON=401 > > 07-11-2014 12:17:10.480 UTC Call-Disconnected: CALL_ID=2010000000/// > [email protected] <mailto:CALL_ID=2010000000///[email protected]> > REASON=408 > > 07-11-2014 12:24:12.694 UTC Call-Disconnected: CALL_ID=2010000000/// > [email protected] <mailto:CALL_ID=2010000000///[email protected]> > REASON=401 > > 07-11-2014 12:24:12.776 UTC Call-Disconnected: CALL_ID=2010000001/// > [email protected] <mailto:CALL_ID=2010000001///[email protected]> > REASON=401 > > 07-11-2014 12:32:02.353 UTC Call-Connected: > FROM=sip:[email protected] <mailto:[email protected]> > TO=sip:[email protected] <mailto:[email protected]> > CALL_ID=2010000000-1.000000///[email protected] <mailto:CALL_ID > =2010000000-1.000000///[email protected]> > > 07-11-2014 12:32:02.358 UTC Call-Connected: > FROM=sip:[email protected] <mailto:[email protected]> > TO=sip:[email protected] <mailto:[email protected]> > CALL_ID=2010000000-1.000000///[email protected] <mailto:CALL_ID > =2010000000-1.000000///[email protected]> > > 07-11-2014 12:35:28.383 UTC Call-Disconnected: CALL_ID=2010000000/// > [email protected] <mailto:CALL_ID=2010000000///[email protected]> > REASON=401 > > 07-11-2014 12:35:28.445 UTC Call-Disconnected: CALL_ID=2010000001/// > [email protected] <mailto:CALL_ID=2010000001///[email protected]> > REASON=401 > > 07-11-2014 12:37:27.355 UTC Call-Disconnected: > CALL_ID=NDk3MjIxZjlmNjU4MWM2NGIxYmY5NDE3NWMxMGE4MTk. REASON=403 > > 07-11-2014 12:37:30.219 UTC Call-Disconnected: > CALL_ID=OWEwNTRkYzFhYWRlZjMwZDRhNjM2NWY0OTgxZDRjNDA. REASON=403 > > 07-11-2014 12:37:32.211 UTC Call-Disconnected: > CALL_ID=NTNiOWE0ZmUzYTYxMjQxOTk4MWU5MTUwZTcyZDI5OTA. REASON=403 > > 07-11-2014 12:37:37.552 UTC Call-Disconnected: > CALL_ID=OWJlMDc5ZTU1NDk3YWZiODhiZjkxNGIyZWFhZjljZmQ. REASON=403 > > 07-11-2014 12:37:39.583 UTC Call-Disconnected: > CALL_ID=NGQwZTM2OWNmNTUyZTA5Yjc4ZjZiNDk4ZWEwMGVhYjI. REASON=403 > > 07-11-2014 12:37:48.224 UTC Call-Disconnected: > CALL_ID=OTYxNDNjYmJhZTJlZTVhYjdkOGZjMTE4ZDY0MDM2MGY. REASON=403 > > 07-11-2014 12:38:43.427 UTC Call-Disconnected: > CALL_ID=ZTFiNThlODM5YzAxNDRiZjdkZTA5NDkyZjdiZDI5NmQ. REASON=403 > > > > sprout: > > > > 07-11-2014 12:02:37.679 UTC Registration: > USER_URI=sip:[email protected] <mailto:[email protected]> > BINDING_ID=<urn:uuid:00000000-0000-0000-0000-000000000001>:1 CONTACT_URI= > sip:[email protected] > <mailto:[email protected]>:29895;transport=UDP;ob > EXPIRES=3600 > > 07-11-2014 12:02:37.695 UTC Registration: > USER_URI=sip:[email protected] <mailto:[email protected]> > BINDING_ID=<urn:uuid:00000000-0000-0000-0000-000000000001>:1 CONTACT_URI= > sip:[email protected] > <mailto:[email protected]>:29895;transport=UDP;ob > EXPIRES=3600 > > 07-11-2014 12:07:27.709 UTC Registration: > USER_URI=sip:[email protected] <mailto:[email protected]> > BINDING_ID=<urn:uuid:00000000-0000-0000-0000-000000000001>:1 CONTACT_URI= > sip:[email protected] > <mailto:[email protected]>:29895;transport=UDP;ob > EXPIRES=3600 > > 07-11-2014 12:07:27.718 UTC Registration: > USER_URI=sip:[email protected] <mailto:[email protected]> > BINDING_ID=<urn:uuid:00000000-0000-0000-0000-000000000001>:1 CONTACT_URI= > sip:[email protected] > <mailto:[email protected]>:29895;transport=UDP;ob > EXPIRES=3600 > > 07-11-2014 12:12:17.735 UTC Registration: > USER_URI=sip:[email protected] <mailto:[email protected]> > BINDING_ID=<urn:uuid:00000000-0000-0000-0000-000000000001>:1 CONTACT_URI= > sip:[email protected] > <mailto:[email protected]>:29895;transport=UDP;ob > EXPIRES=3600 > > 07-11-2014 12:12:17.745 UTC Registration: > USER_URI=sip:[email protected] <mailto:[email protected]> > BINDING_ID=<urn:uuid:00000000-0000-0000-0000-000000000001>:1 CONTACT_URI= > sip:[email protected] > <mailto:[email protected]>:29895;transport=UDP;ob > EXPIRES=3600 > > 07-11-2014 12:17:07.756 UTC Registration: > USER_URI=sip:[email protected] <mailto:[email protected]> > BINDING_ID=<urn:uuid:00000000-0000-0000-0000-000000000001>:1 CONTACT_URI= > sip:[email protected] > <mailto:[email protected]>:29895;transport=UDP;ob > EXPIRES=3600 > > 07-11-2014 12:24:12.733 UTC Registration: > USER_URI=sip:[email protected] <mailto:[email protected]> > BINDING_ID=<urn:uuid:00000000-0000-0000-0000-000000000001>:1 CONTACT_URI= > sip:[email protected] > <mailto:[email protected]>:57263;transport=UDP;ob > EXPIRES=3600 > > 07-11-2014 12:24:12.796 UTC Registration: > USER_URI=sip:[email protected] <mailto:[email protected]> > BINDING_ID=<urn:uuid:00000000-0000-0000-0000-000000000001>:1 CONTACT_URI= > sip:[email protected] > <mailto:[email protected]>:57263;transport=UDP;ob > EXPIRES=3600 > > 07-11-2014 12:29:02.832 UTC Registration: > USER_URI=sip:[email protected] <mailto:[email protected]> > BINDING_ID=<urn:uuid:00000000-0000-0000-0000-000000000001>:1 CONTACT_URI= > sip:[email protected] > <mailto:[email protected]>:57263;transport=UDP;ob > EXPIRES=3600 > > 07-11-2014 12:29:02.857 UTC Registration: > USER_URI=sip:[email protected] <mailto:[email protected]> > BINDING_ID=<urn:uuid:00000000-0000-0000-0000-000000000001>:1 CONTACT_URI= > sip:[email protected] > <mailto:[email protected]>:57263;transport=UDP;ob > EXPIRES=3600 > > 07-11-2014 12:35:28.406 UTC Registration: > USER_URI=sip:[email protected] <mailto:[email protected]> > BINDING_ID=<urn:uuid:00000000-0000-0000-0000-000000000001>:1 CONTACT_URI= > sip:[email protected] > <mailto:[email protected]>:43377;transport=UDP;ob > EXPIRES=3600 > > 07-11-2014 12:35:28.466 UTC Registration: > USER_URI=sip:[email protected] <mailto:[email protected]> > BINDING_ID=<urn:uuid:00000000-0000-0000-0000-000000000001>:1 CONTACT_URI= > sip:[email protected] > <mailto:[email protected]>:43377;transport=UDP;ob > EXPIRES=3600 > > > > clearwater-sip-stress: > > > > sipp: The following events occured: > > 2014-11-07 12:52:58:406 1415361178.406342: Call-Id: > [email protected] <mailto:[email protected]>, receive timeout on > message Call Load Test:34 without label to jump to (ontimeout attribute): > aborting call. > > 2014-11-07 13:17:08:878 1415362628.878665: The minor watchdog > timer 500ms has been tripped (535), 120 trips remaining.. > > 2014-11-07 13:17:10:483 1415362630.483070: Aborting call on > unexpected message for Call-Id '[email protected] <mailto: > [email protected]>': while expecting '200' (index 12), received > 'SIP/2.0 408 Request Timeout > > Via: SIP/2.0/UDP 192.168.3.62:29895 > ;rport=29895;received=10.40.7.169;branch=z9hG4bK-27529-2-11-2010000000-8.000000 > > Call-ID: 2010000000///[email protected] <mailto:2010000000/// > [email protected]> > > From: <sip:[email protected] <sip:[email protected] > >>;tag=27529SIPpTag002 > > To: <sip:[email protected] <sip:[email protected] > >>;tag=z9hG4bK-27529-2-11-2010000000-8.000000 > > CSeq: 9 REGISTER > > Content-Length: 0 > > > > '. > > 2014-11-07 13:27:09:230 1415363229.230236: Resetting watchdog > timer trigger counts, as it has not been triggered in over 600351ms.. > > 2014-11-07 13:32:14:384 1415363534.384905: Call-Id: > [email protected] <mailto:[email protected]>, receive timeout on > message Call Load Test:34 without label to jump to (ontimeout attribute): > aborting call. > > > > > > > > > >> Il giorno 06/nov/2014, alle ore 16:34, Alex Hockey < > [email protected] <mailto:[email protected]>> ha > scritto: > >> > >> Hi there, > >> > >> I am surprised that modifying the bulk provisioning script made a > difference. The two database rows that you mentioned are functionally > equivalent – the `known_preferred` field is no longer used, and Clearwater > will use a quality of protection of “auth if ” `digest_qop` is not filled > in. > >> > >> Are you able to register a SIP phone using one of the bulk provisioned > numbers? If so, are you able to register two phones to different bulk > provisioned numbers and make a call between them. > >> > >> If this works, then the problem definitely lies with the SIPp script > somewhere. The next step is to reduce the number of virtual subscribers > that the SIPp nodes create, and get some logs from the running system. To > do this: > >> - Turn on debug logging on your sprout node. By default the > logging level is set to log level 2, which only includes errors and very > high level events. To enable debug logs, change the log level to 5 by > writing log_level=5 to /etc/clearwater/user_settings (creating it if it > doesn't exist already), and restart sprout (using service sprout stop - > monit automatically restarts it). Sprout will now be writing debug logs to > /var/log/sprout/sprout_*. > >> - Edit /usr/share/clearwater/infrastructure/scripts/sip-stress, > and set the `count` variable at the top of the file to `2`. > >> - Restart SIP stress by running `sudo service > clearwater-sip-stress stop; sudo service clearwater-infrastructure restart; > sudo service clearwater-sip-stress start` > >> > >> Could you try this out and share the output from the most recent logs > from /var/log/sprout and /var/log/clearwater-sip-stress > >> > >> Thanks, > >> Alex. > >> > >> > >> From: Doctor Mescaline [mailto:[email protected] <mailto: > [email protected]>] > >> Sent: 05 November 2014 15:46 > >> To: Alex Hockey > >> Cc: [email protected] <mailto: > [email protected]> > >> Subject: Re: [Clearwater] Benchmarking tool > >> > >> Hi Alex, > >> > >> every node works well with ‘sudo monit status’. The Clearwater Live > Tests pass, maybe because they perform “sigh up” with ellis to get > authorization. > >> > >> From my understanding, what the “provision subscriber” does is to > directly inject the authorization of the numbers into the database, hence > skipping the sign up step on ellis. > >> > >> > >> If I run the “provision subscriber” I get this entry > >> > >> private_id | digest_ha1 | > digest_qop | digest_realm | known_preferred > >> > ----------------------------+----------------------------------+------------+-----------------+----------------- > >> [email protected] <mailto:[email protected]> | > bcdbbf195231a7c2285b713b373aa70a | null | demo.clearwater | > null > >> > >> > >> So I modified the live test suite to register with existing numbers and > passwords in such records instead of signing up with ellis. > >> But it didn’t work util I notice that the record for an authorized > number should be like: > >> > >> private_id | digest_ha1 | > digest_qop | digest_realm | known_preferred > >> > ----------------------------+----------------------------------+------------+-----------------+----------------- > >> [email protected] <mailto:[email protected]> | > bcdbbf195231a7c2285b713b373aa70a | auth | demo.clearwater | > True > >> > >> Then I modified the provision script to form up the records in above > way and this time the test live suite works well with basic calls, which > means that such numbers are available for register-make a call-unregister. > >> But under such configuration we still get the stress test node failed > (0 success call in the output csv file) with messages like: > >> > >> 2014-11-04 07:15:38:733 1415081738.733320: Call-Id: > [email protected] <mailto:[email protected]>, receive timeout > on message Call Load Test:19 without label to jump to (ontimeout > attribute): aborting call. > >> > >> The logs of other nodes seem normal except the bono node showing some > 503. > >> > >> Thanks > >> > >> > >> Il giorno 05/nov/2014, alle ore 12:59, Alex Hockey < > [email protected] <mailto:[email protected]>> ha > scritto: > >> > >> Hi there, > >> > >> Provisioning the subscribers after turning up the SIPp node should work. > >> > >> You are seeing two types of “disconnected” logs from Bono: > >> - The 401s are expected. The first time a client registers with > Clearwater it will not provide authentication credentials. Sprout rejects > the REGISTER message with a 401 response, containing an authentication > challenge. The client then sends another REGISTER with credentials. > >> - The 503s are not expected. I suspect these are responses to the 2nd, > authenticated REGISTER message. They suggest a part of your deployment is > not running properly. > >> > >> The first step is to check that all the Clearwater processes are > running successfully. Run `sudo monit status` on a node to check this, and > look out for any entries that are reported as failed or non-existent. If > you are using an all-in-one node, just run `sudo monit status` from that > node. If you are running a multi-node deployment, run the command on all > your nodes. > >> > >> Alex. > >> > >> From: Doctor Mescaline [mailto:[email protected] <mailto: > [email protected]>] > >> Sent: 04 November 2014 12:17 > >> To: Alex Hockey > >> Cc: [email protected] <mailto: > [email protected]> > >> Subject: Re: [Clearwater] Benchmarking tool > >> > >> Hi Alex, > >> > >> I already did it and calls were always disconnected with REASON=401 or > REASON=503. > >> > >> I created the new SIPp node and then bulked provision subscribers, > while you suggest to bulk provision subscribers and then create the new > SIPp node. > >> > >> Is this order important? > >> > >> > >> Thanks > >> > >> Il giorno 04/nov/2014, alle ore 12:47, Alex Hockey < > [email protected] <mailto:[email protected]>> ha > scritto: > >> > >> Hi there, > >> > >> I suspect the problem here is that you haven’t provisioned any > subscribers on your deployment. When the SIPp nodes try to register, > clearwater rejects their requests with a 403 Forbidden. > >> > >> The stress testing wiki page ( > https://github.com/Metaswitch/clearwater-docs/wiki/Clearwater-stress-testing > < > https://github.com/Metaswitch/clearwater-docs/wiki/Clearwater-stress-testing>) > was missing a “provision subscribers” step. I have updated it so that is > contains a link to the instructions for how to do this. It is a fairly > simple process, that just involves running some scripts on your homer and > homestead nodes. > >> > >> I hope this helps. We haven’t had many people running stress against a > manually installed system before, and you’ve helped us find some gaps in > our documentation! > >> > >> Thanks, > >> Alex. > >> > >> From: The Doctor [mailto:[email protected] <mailto: > [email protected]>] > >> Sent: 31 October 2014 15:58 > >> To: Alex Hockey > >> Subject: Re: [Clearwater] Benchmarking tool > >> > >> Hi Alex, > >> > >> I got it. I have configured the file and installed the package, but it > 'is not able to successfully run the scenario. When running the stress > test, I got these message in the bono node > >> > >> UTC Call-Disconnected: CALL_ID=7000069028///[email protected] > <mailto:[email protected]> REASON=403 > >> > >> Do you think the CALL_ID format is correct? > >> > >> Thanks > >> > >> > >> > >> 2014-10-31 14:03 GMT+01:00 Alex Hockey <[email protected] > <mailto:[email protected]>>: > >> Hi there, > >> > >> I assume that you’ve set up a Clearwater deployment using the Manual > Install instructions ( > https://github.com/Metaswitch/clearwater-docs/wiki/Manual-Install < > https://github.com/Metaswitch/clearwater-docs/wiki/Manual-Install>). The > SIP stress node should be a completely new virtual machine. You should > bootstrap this machine in the same way you bootstrapped the nodes as part > of setting up the deployment (this bit wasn’t clear in the page I pointed > you at). You can then install clearwater-sip-stress as a normal debian > package by running `sudo apt-get install clearwater-sip-stress`. > >> > >> Alex. > >> > >> p.s. Please could you CC the list on responses? That way other members > of the community benefit from the discussion :-). > >> > >> From: The Doctor [mailto:[email protected] <mailto: > [email protected]>] > >> Sent: 31 October 2014 11:02 > >> To: Alex Hockey > >> Subject: Re: [Clearwater] Benchmarking tool > >> > >> Hi Alex, > >> > >> I have read the wiki, but I don't understand where I can downaload the > clearwater-sip-stress Debian package and in which node I have to edit the > /etc/clearwater/config. > >> > >> Thanks > >> > >> 2014-10-31 11:07 GMT+01:00 Alex Hockey <[email protected] > <mailto:[email protected]>>: > >> Hi there, > >> > >> In the Project Clearwater team we use our clearwater-sip-stress package > for stress testing and benchmarking. This is built on SIPp ( > http://sipp.sourceforge.net/ <http://sipp.sourceforge.net/>). There is > more information about clearwater-sip-stress on the wiki ( > https://github.com/Metaswitch/clearwater-docs/wiki/Clearwater-stress-testing > < > https://github.com/Metaswitch/clearwater-docs/wiki/Clearwater-stress-testing > >). > >> > >> By default clearwater-sip-stress runs a fairly simple scenario, where > simulated endpoints REGISTER, then periodically re-REGISTER and make simple > calls. However you can edit the SIPp scenario file to emulate different > behaviour. > >> > >> I'm not sure if this is what you're after, but I hope it helps. Other > users on the list may have better suggestions though! > >> > >> Alex. > >> > >> -----Original Message----- > >> From: [email protected] <mailto: > [email protected]> [mailto: > [email protected] <mailto: > [email protected]>] On Behalf Of The Doctor > >> Sent: 31 October 2014 09:12 > >> To: [email protected] <mailto: > [email protected]> > >> Subject: [Clearwater] Benchmarking tool > >> > >> Hi list, > >> > >> I am looking for a benchmarking tool in order to emulates users > behavior according to variours workload patterns. Is there any tool or > Step-by-Step guide to do that? > >> > >> Thanks > >> _______________________________________________ > >> Clearwater mailing list > >> [email protected] <mailto: > [email protected]> > >> http://lists.projectclearwater.org/listinfo/clearwater < > http://lists.projectclearwater.org/listinfo/clearwater> > > _______________________________________________ > Clearwater mailing list > [email protected] > http://lists.projectclearwater.org/listinfo/clearwater > _______________________________________________ Clearwater mailing list [email protected] http://lists.projectclearwater.org/listinfo/clearwater
