[Kamailio-Users] R: R: Fast lock loop with arm
Hello,?xml:namespace prefix = o ns = urn:schemas-microsoft-com:office:office
/
I have tried a lot of times with different processes, but backtrace shows
always only this.
Daniele
-Messaggio originale-
Da: Daniel-Constantin Mierla [mailto:mico...@gmail.com]
Inviato: venerdì 26 marzo 2010 19.40
A: Zappasodi Daniele
Cc: users@lists.kamailio.org; sr-dev
Oggetto: Re: R: [Kamailio-Users] Fast lock loop with arm
Hello,
On 3/26/10 4:13 PM, Zappasodi Daniele wrote:
Hello,
this is what I get with gdb:
(gdb) bt full
#0 0x40137e54 in sched_yield () from /usr/local/lib2/libc.so.6
No symbol table info available.
I don't think that it can help, but I am not able to load the symbol table for
openser on the server.
hmm, strange. Did you try with many processes? Sam result in the backtrace?
Cheers,
Daniel
thanks,
Daniele
-Messaggio originale-
Da: Daniel-Constantin Mierla [ mailto:mico...@gmail.com]
Inviato: mercoledì 24 marzo 2010 16.47
A: Zappasodi Daniele
Cc: users@lists.kamailio.org; sr-dev
Oggetto: Re: [Kamailio-Users] Fast lock loop with arm
Hello,
what version of kamailio do you use?
Can you grab a backtrace with gdb once you get this high cpu usage? Spot one of
the processes, take the pid and do:
gdb /path/to/kamailio _pid_
Then:
bt
You should see the bactrace of executed function getting to deadloc. Make sure
you get at least one SIP worker process backtrace (do kamctl ps to see the type
of kamailio process).
Cheers,
Daniel
On 3/24/10 4:29 PM, Zappasodi Daniele wrote:
Hi,
I have a big problem with fast lock mutex on arm CPU:
sometimes I find one or more children that go in an infinite loop, in the while
loop of get_lock function.
They remain in Run for long time (sometimes hours) and openser keeps 100% CPU.
Now I have changed the functions get_lock and tsl in order to obtain more info
and
I observe that (if and) when the process finish the cycle, it have done a big
amount of cycles.
This the log with my added info:
Mar 26 20:29:08 SAM-IP openser[1645]: NOTICE:tm:_lock: ret 1 (cycle)
Mar 26 20:29:51 SAM-IP openser[1645]: NOTICE:tm:_lock: ret 1 (cycle)
Mar 26 20:29:55 SAM-IP openser[1645]: NOTICE:tm:_lock: ret 1 (cycle)
Mar 26 20:29:55 SAM-IP openser[1645]: NOTICE:tm:t_retransmit_reply: MYTM lock
[process in loop]
Mar 26 20:33:46 SAM-IP openser[1645]: NOTICE:tm:_lock: ret 59374917 (cycle)
[after 4 minutes and 59374917 cycles, this is an example with a short loop]
Mar 26 20:33:46 SAM-IP openser[1645]: NOTICE:tm:t_retransmit_reply: MYTM lock
done
I'm not able to recognize the call flow that generates the loop (I know only
that it always happens when t_retransmit_reply calls LOCK_REPLIES),
but I urgently need a work around to escape from the loop.
This is the get_lock function with my counter j:
inline static int get_lock(fl_lock_t* lock)
{
#ifdef ADAPTIVE_WAIT
int i=ADAPTIVE_WAIT_LOOPS;
int j=1;/* my change /
#endif
while(tsl(lock)){
#ifdef BUSY_WAIT
#elif defined ADAPTIVE_WAIT
j++;/* my change /
if (i0) i--;
else sched_yield();
#else
sched_yield();
#endif
}
return(j);
}
Can I break the lock when my counter reaches a big value?
Should I call the unlock function after the break?
which value can be considered too big?
Thanks,
Daniele
** The
information in this message is confidential and may be legally privileged. It
is intended solely for the addressee. Access to this message by anyone else is
unauthorized. If you are not the intended recipient, any disclosure, copying,
or distribution of the message, or any action or omission taken by you in
reliance on it, is prohibited and may be unlawful. Please immediately contact
the sender if you have received this message in error.
**
___
Kamailio (OpenSER) - Users mailing list
Users@lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
--
Daniel-Constantin Mierla * http://www.asipto.com/
** The
information in this message is confidential and may be legally privileged. It
is intended solely for the addressee. Access to this message by anyone else is
unauthorized. If you are not the intended recipient, any disclosure, copying,
or distribution of the message, or any action or omission taken by you in
reliance on it, is prohibited and may be unlawful. Please immediately contact
the sender if you have received this message in error.
**
--
Daniel-Constantin Mierla * http://www.asipto.com
Re: [Kamailio-Users] R: R: Fast lock loop with arm
Hello,
On 3/29/10 8:53 AM, Zappasodi Daniele wrote:
Hello,
I have tried a lot of times with different processes, but backtrace
shows always only this.
this is really strange. do you use mi_fifo? if yes, when you start
openser run 'openserctl ps'
Spot a udp worker process, the mi fifo process, the main process and the
timer process. when the locking happens, attach to each of them and get
the backtrace.
Thanks,
Daniel
Daniele
-Messaggio originale-
*Da:* Daniel-Constantin Mierla [mailto:mico...@gmail.com]
*Inviato:* venerdì 26 marzo 2010 19.40
*A:* Zappasodi Daniele
*Cc:* users@lists.kamailio.org; sr-dev
*Oggetto:* Re: R: [Kamailio-Users] Fast lock loop with arm
Hello,
On 3/26/10 4:13 PM, Zappasodi Daniele wrote:
Hello,
this is what I get with gdb:
(gdb) bt full
#0 0x40137e54 in sched_yield () from /usr/local/lib2/libc.so.6
No symbol table info available.
I don't think that it can help, but I am not able to load the
symbol table for openser on the server.
hmm, strange. Did you try with many processes? Sam result in the
backtrace?
Cheers,
Daniel
thanks,
Daniele
-Messaggio originale-
*Da:* Daniel-Constantin Mierla [mailto:mico...@gmail.com]
*Inviato:* mercoledì 24 marzo 2010 16.47
*A:* Zappasodi Daniele
*Cc:* users@lists.kamailio.org; sr-dev
*Oggetto:* Re: [Kamailio-Users] Fast lock loop with arm
Hello,
what version of kamailio do you use?
Can you grab a backtrace with gdb once you get this high cpu
usage? Spot one of the processes, take the pid and do:
gdb /path/to/kamailio _pid_
Then:
bt
You should see the bactrace of executed function getting to
deadloc. Make sure you get at least one SIP worker process
backtrace (do kamctl ps to see the type of kamailio process).
Cheers,
Daniel
On 3/24/10 4:29 PM, Zappasodi Daniele wrote:
Hi,
I have a big problem with fast lock mutex on arm CPU:
sometimes I find one or more children that go in an infinite
loop, in the while loop of get_lock function.
They remain in Run for long time (sometimes hours) and
openser keeps 100% CPU.
Now I have changed the functions get_lock and tsl in order
to obtain more info and
I observe that (if and) when the process finish the cycle,
it have done a big amount of cycles.
This the log with my added info:
Mar 26 20:29:08 SAM-IP openser[1645]: NOTICE:tm:_lock: ret 1
(cycle)
Mar 26 20:29:51 SAM-IP openser[1645]: NOTICE:tm:_lock: ret 1
(cycle)
Mar 26 20:29:55 SAM-IP openser[1645]: NOTICE:tm:_lock: ret 1
(cycle)
Mar 26 20:29:55 SAM-IP openser[1645]:
NOTICE:tm:t_retransmit_reply: MYTM lock
[process in loop]
Mar 26 20:33:46 SAM-IP openser[1645]: NOTICE:tm:_lock: ret
59374917 (cycle)
[after 4 minutes and 59374917 cycles, this is an example
with a short loop]
Mar 26 20:33:46 SAM-IP openser[1645]:
NOTICE:tm:t_retransmit_reply: MYTM lock done
I'm not able to recognize the call flow that generates the
loop (I know only that it always happens when
t_retransmit_reply calls LOCK_REPLIES),
but I urgently need a work around to escape from the loop.
This is the get_lock function with my counter j:
inline static int get_lock(fl_lock_t* lock)
{
#ifdef ADAPTIVE_WAIT
int i=ADAPTIVE_WAIT_LOOPS;
int j=1;/* my
change /
#endif
while(tsl(lock)){
#ifdef BUSY_WAIT
#elif defined ADAPTIVE_WAIT
j++;/* my change /
if (i0) i--;
else sched_yield();
#else
sched_yield();
#endif
}
return(j);
}
Can I break the lock when my counter reaches a big value?
Should I call the unlock function after the break?
which value can be considered too big?
Thanks,
Daniele
**
The information in this message is confidential and may be
legally privileged. It is intended solely for the addressee.
Access to this message by anyone else is unauthorized. If
you are not the intended recipient, any disclosure, copying,
or distribution of the message, or any action or omission
taken by you in reliance on it, is prohibited and may be
unlawful. Please immediately contact the sender if you have
received this message in error
