Actually more likely it has been both. The root problem lies in the timer subsystem and may be amplified by other troubles (or amplify those).
-jiri At 01:35 30/03/2007, T.R. Missner wrote: >FYI All > >This turned out to be a database write ( acc ) that was blocking due to a raid >card problem. > > > >T.R. Missner wrote: >>Is it possible the locked state I am seeing with openser leads to the >>"detached" timer? >>Since the "detached" timer is a race, it would make sense to see the race >>condition after openser locks up and messages buffer up in the stack. >>When a bunch of messages are processed all at once by multiple threads the >>race condition would occur. >>Does this make sense? >> >>Maybe I have been focusing on the wrong place. >> >>Ignoring the "detached" timer what could cause openser to hang for a couple >>seconds then clear every 5 - 10 minutes? >> >>Ideas? >> >>We are seeing this on 3 different productions servers. >> >>Thanks >> >>TR >> >>using openser1.1.1 >> >> >> >>T.R. Missner wrote: >>>Bogdan, >>> >>>I have been chasing this for days and done lots of debugging. >>>using 1.1.1 >>>While looking at the network trace at the time of these messages ( I usually >>>see at least 5 in a row with differing hex values ) I see many incoming >>>packets coming into the box and no response from the proxy for somewhere >>>between 5 - 10 seconds, then a flood a responses from the proxy. >>>I can email you a sample pcap file if you like. >>>As part of my debugging I forced a 100 reply at the very top of my cfg file. >>>The forced 100 was not sent during the locked up time leading me to believe >>>openser was not processing incoming packets. >>>I have now seen this on multiple servers in different locations. Likely a >>>particular customer call flow is causing this but I have not been able to >>>pin it down to the exact customer. These proxies run pretty fast during the >>>day so finding a pattern leading up the this issue is difficult. What could >>>I add to the Log output to identify the offending sip-callid? Is sip-callid >>>or branch tag or anything similar easily accessible in any of the data >>>structs in timer.c? >>> >>>TR >>> >>>Bogdan-Andrei Iancu wrote: >>>>Hi TR, >>>> >>>>it is race between expire even (from timer) and inserting again on a timer >>>>list. >>>> 1 is the final response timer list (fr_timer) >>>> 3 id the wait timer list (wt_timer) >>>> >>>>I would say there is no way this could leas to a any kind of lock. >>>> >>>>what version are you using? what makes you say it locks? >>>> >>>>regards, >>>>bogdan >>>> >>>>T.R. Missner wrote: >>>>>Does anyone know what causes this? >>>>> >>>>>*/set_timer for 1 list called on a "detached" timer -- ignoring /* >>>>> >>>>>I also see >>>>> >>>>>*/set_timer for 3 list called on a "detached" timer -- ignoring /* >>>>> >>>>> >>>>> >>>>>When this happens Openser seems to lock up for 10 seconds or so. >>>>> >>>>>>From searching it appears this is caused by a race but I am not sure what >>>>>>the race is or why this results in an unresponsive openser instance for >>>>>>multiple seconds. >>>>> >>>>>Transaction expiration racing reply? >>>>> >>>>> >>>>>Desperately need to understand how this could be triggered so I can get >>>>>customer to adjust system. >>>>> >>>>>Any way to adjust? >>>>> >>>>>tried tweaking fr_inv_timer but no joy. >>>>> >>>>> >>>>> >>>>>TR >>>>>------------------------------------------------------------------------ >>>>> >>>>>_______________________________________________ >>>>>Devel mailing list >>>>>[email protected] >>>>>http://openser.org/cgi-bin/mailman/listinfo/devel >>>>> >>> >>>_______________________________________________ >>>Users mailing list >>>[email protected] >>>http://openser.org/cgi-bin/mailman/listinfo/users >> >>_______________________________________________ >>Users mailing list >>[email protected] >>http://openser.org/cgi-bin/mailman/listinfo/users > >_______________________________________________ >Users mailing list >[email protected] >http://openser.org/cgi-bin/mailman/listinfo/users -- Jiri Kuthan http://iptel.org/~jiri/ _______________________________________________ Users mailing list [email protected] http://openser.org/cgi-bin/mailman/listinfo/users
