Daniel,
Got the same issue on 5.3.1 with openssl1.1, debian9.
After 3 working days of tests (about ~30-50 wss clients), suddenly we've got
a lot of connections stucked in CLOSE_WAIT state. Kamailio called
sig_alarm_abort() when we try to reboot.
Thanks,
Andrey
--
Sent from:
Hello,
for deadlock issue with libssl 1.1 an workaround with a preloaded
library was made available quite some time ago:
https://github.com/kamailio/kamailio/tree/master/src/modules/tls/utils/openssl_mutex_shared
Recently that code was integrated in the core, so the preloaded library
is not
Hi Daniel,
I hope you are well. Do you have any updates on this issue? Did you get any
response on openssl mailing list? Thank you!
With kind regards,
Jurijs
On Mon, Apr 1, 2019 at 11:55 AM Daniel-Constantin Mierla
wrote:
> Hello,
>
> an update on this issue -- I spent a bit of time looking
Tks Daniel,
I have installed the workaround.
lsof seems to indicate that I have installed and
pre-loaded openssl_mutex_shared.so correctly.
I will let you know if I see the issue again.
Tks!
Aymeric
Le lun. 20 mai 2019 à 09:49, Daniel-Constantin Mierla a
écrit :
> Hello,
>
> this kind of
>> http://www.commend.com <http://www.commend.com/>
>>
>> *Security and Communication by Commend
>>
>> *FN 178618z | LG Salzburg
>>
>>
>>
>> *Von: *sr-users
>> <mailto:sr-users-boun...@lists.kamaili
Software-Development
>
>
> *COMMEND INTERNATIONAL GMBH *A-5020 Salzburg, Saalachstraße 51
> http://www.commend.com
>
>
>
> *Security and Communication by Commend *FN 178618z | LG Salzburg
>
>
>
> *Von: *sr-users
> im Auftrag von Daniel-Constantin
> Mierla
LG Salzburg
>
>
>
> *Von: *sr-users im Auftrag von
> Daniel-Constantin Mierla
> *Antworten an: *"mico...@gmail.com" , "Kamailio
> (SER) - Users Mailing List"
> *Datum: *Montag, 15. April 2019 um 09:07
> *An: *Aymeric Moizard , "Kamailio (SER) - Users
lzburg
Von: sr-users im Auftrag von
Daniel-Constantin Mierla
Antworten an: "mico...@gmail.com" , "Kamailio (SER) - Users
Mailing List"
Datum: Montag, 15. April 2019 um 09:07
An: Aymeric Moizard , "Kamailio (SER) - Users Mailing List"
Betreff: Re: [SR-Users] K
Hello,
I think one possibility to reproduce the issue would be to create a
scenario when same connection is wanted at the same time, when the first
process that gets the lock on it needs a bit more time to execute. Not
sure how to create the case, maybe something like:
- enable onsend_route
HI Daniel,
I have received your request and have added it to my TODO list...
Unfortunatly, no much time currently. I will certainly do it later, but
cannot give any delay for it.
Also, I would really like to understand how to "generate" the issue.
(I think I had the issue only once or twice
Hello Aymeric,
would you be able to test with tls module compiled against libssl 1.1
and using the pre-loaded shared object workaround?
*
https://github.com/kamailio/kamailio/tree/master/src/modules/tls/utils/openssl_mutex_shared
You should be able to use it with any version, no need to test
Hi,
(X-posted to sr-dev as this is getting into the nitty gritty)
As a short-term workaround for this, I've been playing with the
preloaded library approach to hijack the pthread mutex calls and force
them to provide process-shared mutexes. AFAICT this seems to be working
and only has the
Hello,
an update on this issue -- I spent a bit of time looking at
libssl/libcrypto library and the problem can be the type of mutexes they
use now internally starting with v1.1, respectively the pthread mutex.
They are not process shared and kamailio is a multi-process application,
working with
Hi Andrew,
yes, with openssl 1.0.2 Kamailio is now up and running since five
days. Looks good so far.
Kristijan
Am Do., 28. März 2019 um 11:09 Uhr schrieb Andrew Pogrebennyk
:
>
> On 3/26/19 3:52 PM, Kristijan Vrban wrote:
> >> Just curious, did you get to compile with OpenSSL 1.0 and test?
> >
On 3/26/19 3:52 PM, Kristijan Vrban wrote:
>> Just curious, did you get to compile with OpenSSL 1.0 and test?
> Just compiled with OpenSSL 1.0 . Gone test now.
Kristijan,
any new occurrences since you have recompiled kamailio with openssl 1.0?
Regards,
Andrew
Hello,
yep, locking there is expected, as listing the tls connections wait for
no other processes to change the content of internal tls connection
structures. So it is a side effect of libssl/libcrypto getting stuck and
the other processing waiting for it to move one. I have the Kamailio
training
Hi All,
I was debugging a TCP issue (most probably, I may start a thread for this
question).
I was trying to get some info for TCP and TLS.
I typed:
$> sudo kamctl rpc tls.list
And waited for a while until... I realized that my User-Agent,
connected with TCP was not able to register any
> Just curious, did you get to compile with OpenSSL 1.0 and test?
Just compiled with OpenSSL 1.0 . Gone test now.
Am Di., 26. März 2019 um 15:40 Uhr schrieb Joel Serrano :
>
> Just curious, did you get to compile with OpenSSL 1.0 and test?
>
> On Tue, Mar 26, 2019 at 06:12 Kristijan Vrban
Just curious, did you get to compile with OpenSSL 1.0 and test?
On Tue, Mar 26, 2019 at 06:12 Kristijan Vrban wrote:
> And again one more kamctl trap file where
>
> set_reply_no_connect was set.
>
> Am Di., 26. März 2019 um 08:53 Uhr schrieb Kristijan Vrban
> :
> >
> > Attached also the output
> Have you done a test with tools such as sipp, or was this happening
> after a while, with usual phones registering?
Usual variety of devices registering via TLS. But i can not exclude
that some devices displaying behavioural problems.
> Can you list the tcp connections and see if they are
I looked similar examples when
1) used perl module + perl app in kamailio config;
2) used http_client module and upstream http server return error message
with size about 64Kb.
you can check your config for external server calls. Think this may be
related.
Sergey
пн, 25 мар. 2019 г. в 16:28,
> The solution here is to use set_reply_no_connect()
implemented it. Now the issue has shifted to:
ERROR: [core/tcp_main.c:3959]: handle_new_connect(): maximum
number of connections exceeded: 2048/2048
But not a single TCP connection is active between Kamailio and any
device. Seems this
Yes I agree there... I meant the trigger. I have a feeling your issue is
with TLS, and when it happens it affects the rest...
Give it a try and let us know ;)
On Sat, Mar 23, 2019 at 12:05 Aymeric Moizard wrote:
> Hi Joel,
>
> My issue was that any TCP traffic wasn't working: including TLS. I
Hi Joel,
My issue was that any TCP traffic wasn't working: including TLS. I guess it
could be related to the SSL.1.1 issue.
Tks!
Aymeric
Le ven. 22 mars 2019 à 21:21, Joel Serrano a écrit :
> Hi Aymeric,
>
> Are you sure the issue is with TCP and not strictly related to TLS? I
> highly
To make it easier for you and not have to go through the whole thread, if
you want the TL;DR start here:
https://github.com/kamailio/kamailio/issues/1172#issuecomment-312634272
On Fri, Mar 22, 2019 at 1:19 PM Joel Serrano wrote:
> Hi Aymeric,
>
> Are you sure the issue is with TCP and not
Hi Aymeric,
Are you sure the issue is with TCP and not strictly related to TLS? I
highly suggest you compile with ssl1.0 and give it a try...
If you want to read how I got to that conclusion:
https://github.com/kamailio/kamailio/issues/1172
Hope it helps!
Joel.
On Fri, Mar 22, 2019 at
Hi Daniel,
Tks for the tips.
My traffic does include TLS as well.
For TCP settings:
tcp_connection_lifetime=3600
tcp_async=yes
tcp_rd_buf_size=16384
tcp_accept_no_cl=yes
tcp_max_connections=5
tcp_connect_timeout=7
For TLS:
enable_tls=yes
tls_max_connections=5
I'm using
Do you have pure tcp traffic and facing this issue, or there are
actually tls connections?
What are the values for core parameters related to tcp connect and tcp
send timeouts?
As for restart taking long, see exit_timeout parameter:
*
Hi List,
I want to share that I also met this issue last week with my kamailio 5.2.2.
As far as I was able to see, SIP application were able to "connect()"
with TCP, but my logs wasn't reporting any of the SIP message received
with TCP.
I have an pike right before an xlog showing every incoming
Hello,
based on the trap output I think I could figure out what happened there.
You have tcp_children to very low value (1 or so), the problem is not
actually that one, but the fact that the connection to upstream (the
device/app sending the request) was closed after receiving the request
and
Hello,
setting tcp_children=1 is not a god option for scallability, practically
you set kamailio to process a single tcp message at one time, on high
traffic, that won't work well.
Maybe try to set tcp_children to 2 or 4, that should make an eventual
race appear faster.
Regarding the pid, if it
Can you get file written by `kamctl trap`? It should have the backtrace
for all kamailio processes. You need latest kamailio 5.2.
Also, get the output for: kamctl ps
Cheers,
Daniel
On 14.03.19 13:52, Kristijan Vrban wrote:
> When i attach via gdb to one of the tcp worker, i see this:
>
> (gdb)
When i attach via gdb to one of the tcp worker, i see this:
(gdb) bt
#0 0x7fdaf4d14470 in futex_wait (private=,
expected=1, futex_word=0x7fdaeca92f8c) at
../sysdeps/unix/sysv/linux/futex-internal.h:61
#1 futex_wait_simple (private=, expected=1,
futex_word=0x7fdaeca92f8c) at
Hi, with full debug is see this in log for every incoming TCP SIP request:
Mar 14 12:10:15 kamailio-preview /usr/sbin/kamailio[17940]: DEBUG:
[core/tcp_main.c:3871]: send2child(): WARNING: no free tcp
receiver, connection passed to the least busy one (105)
Mar 14 12:10:15 kamailio-preview
first of all thanks for the feedback. i prepared our system now to run
with debug=3
I hope to see more then then.
Am Mi., 27. Feb. 2019 um 11:53 Uhr schrieb Kristijan Vrban
:
>
> Hi kamailios,
>
> i have a creepy situation with v5.2.1 stable Kamilio. After a day or
> so, Kamailio stop to process
I think need to increase LimitNOFILE in systemd file
https://www.freedesktop.org/software/systemd/man/systemd.exec.html
[Service]
LimitNOFILE=99
.
Sergey
ср, 27 февр. 2019 г., 14:27 Ivaylo Markov :
> Hello,
>
> I believe this issue is related -
>
Hello,
I believe this issue is related -
https://github.com/kamailio/kamailio/issues/1172. We encountered the
problem before and the solution is to link kamailio-tls-modules with
libssl1.0.X instead of libssl1.1.
On 27/02/2019 13:23, Jurijs Ivolga wrote:
> Hi,
>
> Just to add that in my case I
Hi,
Just to add that in my case I had a problem when after some period of time
with a lot of TLS clients(100k+) I got a lot of TCP connections in
CLOSE_WAIT state. When connections in CLOSE_WAIT state hit more then 1k,
then kamailio stopped to receive traffic via TLS, nevertheless UDP at same
when is strace to the kamailio process that is attached to the tcp
port. it get sporadic this:
[], 46, 5000)= 0
epoll_wait(17, [{EPOLLIN, {u32=2692971064, u64=139924137540152}}], 46, 5000) = 1
accept(14, {sa_family=AF_INET, sin_port=htons(59766),
sin_addr=inet_addr("xxx.xx.xxx.xxx")},
Hi,
I experienced something similar on Debian Stretch, nevertheless on Debian
Jessie it worked fine. We use TLS and I was thinking that it is something
to do with SSL libraries, but never had chance to find out. But maybe my
problem was nothing to do with what you just described.
Jurijs
On
40 matches
Mail list logo
