Hi Ellie, Thanks a lot for the response!
On Fri, Oct 10, 2014 at 6:19 AM, Eleanor Merry <[email protected] > wrote: > Hi Lianjie, > > There is a reg_max_expires option that controls the maximum length of a > registration. This defaults to 300 (seconds). To change this, edit > /etc/clearwater/config on each of your sprout nodes and add > reg_max_expires=<time in seconds>, and restart each of your sprouts > (service sprout stop). You can see more about the configuration options at > https://github.com/Metaswitch/clearwater-docs/wiki/Clearwater-Configuration-Options-Reference > . > > I followed the instructions and changed the default setting. It seems work. The 200 response does contain "expires=3600" (I set the reg_max_expires=3600 ). But when I run batch register for 20000 numbers, then invites a few minutes later. It still gives a lot of 403 response. Like this: Register ------------------------------ Scenario Screen -------- [1-9]: Change Screen -- Call-rate(length) Port Total-time Total-calls Remote-host 200.0(0 ms)/1.000s 5060 100.02 s 20000 10.0.0.31:5060(UDP) Call limit reached (-m 20000), 0.000 s period 0 ms scheduler resolution 0 calls (limit 600) Peak was 82 calls, after 43 s 0 Running, 6603 Paused, 0 Woken up 0 dead call msg (discarded) 0 out-of-call msg (discarded) 1 open sockets Messages Retrans Timeout Unexpected-Msg REGISTER ----------> 20000 0 200 <---------- 93 0 0 0 503 <---------- 0 0 0 0 401 <---------- 19907 0 0 0 REGISTER ----------> 19907 0 503 <---------- 0 0 0 0 200 <---------- 19907 0 0 0 ------------------------------ Test Terminated -------------------------------- Invite ------------------------------ Scenario Screen -------- [1-9]: Change Screen -- Call-rate(length) Port Total-time Total-calls Remote-host 165.0(0 ms)/1.000s 5060 70.07 s 9900 10.0.0.31:5060(UDP) Call limit reached (-m 9900), 0.000 s period 0 ms scheduler resolution 0 calls (limit 2475) Peak was 2475 calls, after 42 s 0 Running, 5442 Paused, 0 Woken up 909 dead call msg (discarded) 0 out-of-call msg (discarded) 1 open sockets Messages Retrans Timeout Unexpected-Msg INVITE ----------> 9900 0 503 <---------- 551 0 1076 0 403 <---------- 2954 0 551 0 408 <---------- 0 0 2954 0 100 <---------- 5161 0 0 0 INVITE <---------- 158 0 0 0 100 <---------- 158 0 0 0 INVITE <---------- 5158 0 3 0 180 ----------> 5316 0 503 <---------- 0 0 5 0 408 <---------- 0 0 0 0 180 <---------- 5311 0 0 0 200 ----------> 5311 0 503 <---------- 0 0 7 0 408 <---------- 0 0 0 0 200 <---------- 5304 0 0 0 ACK ----------> 5304 0 503 <---------- 0 0 88 0 408 <---------- 0 0 0 0 ACK <---------- 5216 0 0 0 Pause [ 5000ms] 5216 0 BYE ----------> 5216 0 503 <---------- 0 0 806 0 408 <---------- 0 0 0 0 BYE <---------- 4410 0 0 0 200 ----------> 4410 0 503 <---------- 0 0 7 0 408 <---------- 0 0 0 0 200 <---------- 4403 0 0 0 ------------------------------ Test Terminated -------------------------------- >From my understanding, 403 is usually because of number unregistered. This looks weird to me. > In the scenario in your second question, how is the call failing? Why is > it not receiving a response from the client? Also, what version of > Clearwater are you running? We've recently added timer C support which will > destroy the transaction on timeout, which will fix up the unending > retransmissions. > > Well, it is hard to reproduce this problem. I am running the latest version. Here are some messages I caught using tcpdump after SIPp session ends. And also this usually happens in heavy load testing where thousands of calls are made. And I don't have a way to trace and analyze the where and why the call failed yet. 10:22:40.758816 IP (tos 0x0, ttl 64, id 32111, offset 0, flags [DF], proto UDP (17), length 866) Bono.sip > SIPp-worker.sip: SIP, length: 838 SIP/2.0 180 Ringing Via: SIP/2.0/UDP 10.0.0.11:5060 ;received=10.0.0.11;branch=z9hG4bK-1994-7106-1 Record-Route: <sip:[email protected]:5060;transport=UDP;lr> Record-Route: <sip:10.0.0.31:5058;transport=TCP;lr> Record-Route: <sip:10.0.0.32:5054 ;transport=TCP;lr;service=scscf;billing-role=charge-term> Record-Route: <sip:10.0.0.32:5054 ;transport=TCP;lr;service=scscf;billing-role=charge-orig> Record-Route: <sip:10.0.0.31:5058;transport=TCP;lr> Record-Route: <sip:[email protected]:5060;transport=UDP;lr> From: <sip:[email protected]>;tag=1994caller7106 To: <sip:[email protected]>;tag=1994callee7106 Contact: <sip:[email protected]:5060;transport=UDP> Call-ID: [email protected] CSeq: 1 INVITE Allow: PRACK, INVITE, ACK, BYE, CANCEL, UPDATE, SUBSCRIBE, NOTIFY, REFER, MESSAGE, OPTIONS Content-Length: 0 10:22:40.758877 IP (tos 0xc0, ttl 64, id 44808, offset 0, flags [none], proto ICMP (1), length 576) SIPp-worker > Bono: ICMP SIPp-worker udp port sip unreachable, length 556 IP (tos 0x0, ttl 64, id 32111, offset 0, flags [DF], proto UDP (17), length 866) Bono.sip > SIPp-worker.sip: SIP, length: 838 SIP/2.0 180 Ringing Via: SIP/2.0/UDP 10.0.0.11:5060 ;received=10.0.0.11;branch=z9hG4bK-1994-7106-1 Record-Route: <sip:[email protected]:5060;transport=UDP;lr> Record-Route: <sip:10.0.0.31:5058;transport=TCP;lr> Record-Route: <sip:10.0.0.32:5054 ;transport=TCP;lr;service=scscf;billing-role=charge-term> Record-Route: <sip:10.0.0.32:5054 ;transport=TCP;lr;service=scscf;billing-role=charge-orig> Record-Route: <sip:10.0.0.31:5058;transport=TCP;lr> Record-Route: <sip:[email protected]:5060;transport=UDP;lr> F[|sip] > For your third question, I'm not sure why the ACKs are failing. ACKs > aren't subject to overload control, and if it was due to overload then I'd > expect more of the 200 OK responses to be lost. Can you let me know where > the ACKs are lost (e.g. Sprout->Bono, Bono->Client)? Also how loaded is the > system at this point? > That was my conjecture too. Is there any specific tool I can use to trace the messages by hop. And also it would be great if I can check the delay per hop as well. Thanks, Lianjie > Ellie > > > -----Original Message----- > From: [email protected] [mailto: > [email protected]] On Behalf Of Lianjie Cao > Sent: 07 October 2014 16:51 > To: [email protected] > Cc: Sharma, Puneet > Subject: [Clearwater] Clearwater with Sipp > > Hi, > > We are performing some stress tests using SIPp against a manual > installation of Clearwater. > The xml scenario file we use is a little bit different from call_load2.xml > in clearwater-sip-stress test. We are trying to separate register > unregister from invite and test them separately. > But we ran into some strange behaviors during the our tests. > > 1. No matter what value we set to Expires field in REGISTER message, the > 200 OK response only returns an Expires value no more than 300. So, our > REGISTER can only be valid for no more than 5 minute. Is there a way to > change this default setting? > > 2. If a call failed before 200 OK sent out, Bono keeps sending 180 Ringing > message to our SIPp worker even after the whole SIPp session ends (SIPp > uses UDP by default). And since the session ends, the 180 Ringing from Bono > cannot be delivered to the right port number. This can last for a very long > time. For most times, we have to reboot Bono to resolve this. > > 3. Base on our observation, most of the failed calls are due to timeout on > ACK request from SIPp worker. Attached is one example and our script of ACK > message. > Is there any specific reason for this? Is that because ACK is received and > sent by Sprout, but Sprout is heavily loaded at the moment? > > Thanks, > Lianjie > > ------------------------------ Scenario Screen -------- [1-9]: Change > Screen -- > Call-rate(length) Port Total-time Total-calls Remote-host > 160.0(0 ms)/1.000s 5060 69.96 s 9600 10.0.0.31:5060(UDP) > > Call limit reached (-m 9600), 0.000 s period 0 ms scheduler resolution > 0 calls (limit 2400) Peak was 1120 calls, after 31 s > 0 Running, 4751 Paused, 0 Woken up > 2432 dead call msg (discarded) 27 out-of-call msg (discarded) > > 1 open sockets > > Messages Retrans Timeout > Unexpected-Msg > INVITE ----------> 9600 0 > 503 <---------- 0 0 61 0 > 403 <---------- 0 0 0 0 > 408 <---------- 0 0 0 0 > 100 <---------- 8877 0 0 0 > INVITE <---------- 662 0 0 0 > > 100 <---------- 662 0 0 0 > INVITE <---------- 8877 24 0 0 > > 180 ----------> 9539 24 > 503 <---------- 0 0 0 5 > 408 <---------- 0 0 0 0 > 180 <---------- 9534 0 0 0 > > 200 ----------> 9534 0 > 503 <---------- 0 0 20 0 > 408 <---------- 0 0 0 0 > 200 <---------- 9514 0 0 0 > ACK ----------> 9514 0 > 503 <---------- 0 0 2327 0 > 408 <---------- 0 0 0 0 > ACK <---------- 7187 0 0 0 > Pause [ 5000ms] 7187 0 > BYE ----------> 7187 0 > 503 <---------- 0 0 46 0 > 408 <---------- 0 0 0 0 > BYE <---------- 7141 0 0 0 > > 200 ----------> 7141 0 > 503 <---------- 0 0 0 0 > 408 <---------- 0 0 0 0 > 200 <---------- 7141 0 0 0 > > ------------------------------ Test Terminated > -------------------------------- > > > ----------------------------- Statistics Screen ------- [1-9]: Change > Screen -- > Start Time | 2014-10-06 20:47:49.142762 1412653669.142762 > > Last Reset Time | 2014-10-06 20:48:59.108387 1412653739.108387 > > Current Time | 2014-10-06 20:48:59.108452 1412653739.108452 > > -------------------------+---------------------------+------------------ > -------------------------+---------------------------+-------- > Counter Name | Periodic value | Cumulative value > -------------------------+---------------------------+------------------ > -------------------------+---------------------------+-------- > Elapsed Time | 00:00:00:000000 | 00:01:09:965000 > > Call Rate | 0.000 cps | 137.211 cps > > -------------------------+---------------------------+------------------ > -------------------------+---------------------------+-------- > Incoming call created | 0 | 0 > > OutGoing call created | 0 | 9600 > > Total Call created | | 9600 > > Current Call | 0 | > > -------------------------+---------------------------+------------------ > -------------------------+---------------------------+-------- > Successful call | 0 | 7141 > > Failed call | 0 | 2459 > > -------------------------+---------------------------+------------------ > -------------------------+---------------------------+-------- > Call Length | 00:00:00:000000 | 00:00:06:463000 > > ------------------------------ Test Terminated > -------------------------------- > > > > <recv response="200" rrs="true"></recv> > > <!-- > Client #1 sends ACK to Clearwater > --> > <send> > <![CDATA[ > ACK sip:[$peer_dn]@[local_ip]:[local_port];transport=[transport] > SIP/2.0 > Via: SIP/2.0/[transport] > [local_ip]:[local_port];branch=z9hG4bK-[pid]-[call_number]-1 > [routes] > From: <sip:[$my_dn]@[service]>;tag=[pid]caller[call_number] > To: <sip:[$peer_dn]@[service]>;tag=[pid]callee[call_number] > Call-ID: [call_id] > CSeq: [cseq] ACK > Max-Forwards: 70 > Content-Length: 0 > ]]> > </send> > <!-- Contact: <sip:[$my_dn]@[local_ip]:[local_port];transport=[transport]> > --> > <!-- > Client #2 expects ACK from Clearwater > --> > > <recv request="ACK"></recv> > _______________________________________________ > Clearwater mailing list > [email protected] > http://lists.projectclearwater.org/listinfo/clearwater > _______________________________________________ Clearwater mailing list [email protected] http://lists.projectclearwater.org/listinfo/clearwater
