I understood it happened on a load test session, which typically is not done in production, but some test/staging system.
Cheers, Daniel On 18.09.19 16:01, Andrew Chen wrote: > I'll look for the core. > > As part of our release process, I don't think we can release master > branch of Kamailio. > > On Wed, Sep 18, 2019 at 9:54 AM Daniel-Constantin Mierla > <[email protected] <mailto:[email protected]>> wrote: > > Hello, > > did you get core files? If yes, it would be good to see the gdb > bracktraces. > > Have you used the pre-loaded library workaround? > > The code was not backported to branch 5.2 yet, I wanted to wait a > bit and see if there are reports with good/bad results. If you > want to use it, then you have to install nightly builds from > master branch (development version, should be 5.3.0-pre1 at this > moment). > > Cheers, > Daniel > > On 18.09.19 15:26, Andrew Chen wrote: >> Hi Daniel, >> Which apt repository can we get this workaround? Today I >> noticed our Kamailio 5.2.4 cored during one of our load test >> sessions: >> >> Wed Sep 18 09:31:14 2019] kamailio[26997]: segfault at >> 7f8501366000 ip 00007f8501e58585 sp 00007fffd5c84248 error 4 in >> libcrypto.so.1.1[7f8501dd4000+29b000] >> >> [Wed Sep 18 10:12:29 2019] kamailio[27110]: segfault at >> 7f1a8eea1000 ip 00007f1a8f993585 sp 00007ffddc8848c8 error 4 in >> libcrypto.so.1.1[7f1a8f90f000+29b000] >> >> [Wed Sep 18 10:12:43 2019] kamailio[11060]: segfault at 20 ip >> 00007f0a85dac128 sp 00007fff6847aad0 error 4 in >> libcrypto.so.1.1[7f0a85c40000+29b000] >> >> [Wed Sep 18 10:12:44 2019] kamailio[11059]: segfault at f0 ip >> 00007f0a85cc43ea sp 00007fff6847a9e8 error 4 in >> libcrypto.so.1.1[7f0a85c40000+29b000] >> >> [Wed Sep 18 10:13:02 2019] kamailio[11187]: segfault at >> 7f75e06c8000 ip 00007f75e11ba585 sp 00007ffe29b47c88 error 4 in >> libcrypto.so.1.1[7f75e1136000+29b000] >> >> [Wed Sep 18 10:13:17 2019] kamailio[11445]: segfault at >> 7f0f3743f000 ip 00007f0f37f31585 sp 00007ffde54daa38 error 4 in >> libcrypto.so.1.1[7f0f37ead000+29b000] >> >> [Wed Sep 18 10:13:36 2019] kamailio[11644]: segfault at >> 7f804309c000 ip 00007f8043b8e585 sp 00007ffe89fe8228 error 4 in >> libcrypto.so.1.1[7f8043b0a000+29b000] >> >> [Wed Sep 18 10:13:52 2019] kamailio[11799]: segfault at >> 7ff59c7e2000 ip 00007ff59d2d4585 sp 00007ffd69d76bb8 error 4 in >> libcrypto.so.1.1[7ff59d250000+29b000] >> >> [Wed Sep 18 10:13:52 2019] kamailio[11792]: segfault at >> 7ff59c7e2000 ip 00007ff59d2d4585 sp 00007ffd69d76bb8 error 4 in >> libcrypto.so.1.1[7ff59d250000+29b000] >> >> [Wed Sep 18 12:48:01 2019] kamailio[12066]: segfault at >> 7f22dca6d000 ip 00007f22dd55f585 sp 00007fffbc766ab8 error 4 in >> libcrypto.so.1.1[7f22dd4db000+29b000] >> >> [Wed Sep 18 12:48:14 2019] kamailio[16344]: segfault at >> 7f7cca42d000 ip 00007f7ccaf1f585 sp 00007ffe99f16108 error 4 in >> libcrypto.so.1.1[7f7ccae9b000+29b000] >> >> [Wed Sep 18 12:48:32 2019] kamailio[16708]: segfault at >> 7fb309141000 ip 00007fb309c33585 sp 00007ffd07a5f3c8 error 4 in >> libcrypto.so.1.1[7fb309baf000+29b000] >> >> [Wed Sep 18 12:48:49 2019] kamailio[16824]: segfault at f0 ip >> 00007fc59095b3ea sp 00007ffe58a490b8 error 4 in >> libcrypto.so.1.1[7fc5908d7000+29b000] >> >> [Wed Sep 18 12:49:04 2019] kamailio[16956]: segfault at >> 7fcc22e58000 ip 00007fcc2394a585 sp 00007fffa9c52258 error 4 in >> libcrypto.so.1.1[7fcc238c6000+29b000] >> >> [Wed Sep 18 12:49:04 2019] kamailio[16945]: segfault at f0 ip >> 00007fcc2394a3ea sp 00007fffa9c52128 error 4 in >> libcrypto.so.1.1[7fcc238c6000+29b000] >> >> [Wed Sep 18 12:49:16 2019] kamailio[17244]: segfault at >> 7f586ed20000 ip 00007f586f812585 sp 00007ffeae1adcd8 error 4 in >> libcrypto.so.1.1[7f586f78e000+29b000] >> >> [Wed Sep 18 12:49:33 2019] kamailio[17431]: segfault at 8 ip >> 0000562fcc44c5d5 sp 00007ffe825c9dd0 error 4 in >> kamailio[562fcc17c000+4a2000] >> >> [Wed Sep 18 12:49:33 2019] kamailio[17386]: segfault at 10 ip >> 00007f96c3735c3e sp 00007ffe825ca270 error 4 in >> dialog.so[7f96c36d6000+be000] >> >> [Wed Sep 18 12:49:33 2019] kamailio[17423]: segfault at 0 ip >> 0000562fcc3544c0 sp 00007ffe825ca1f0 error 6 in >> kamailio[562fcc17c000+4a2000] >> >> [Wed Sep 18 12:49:33 2019] kamailio[17378]: segfault at 0 ip >> 00007f96c5dd099f sp 00007ffe825c98e0 error 4 in >> tm.so[7f96c5da5000+13f000] >> >> [Wed Sep 18 12:49:33 2019] kamailio[17417]: segfault at 0 ip >> 0000562fcc3544c0 sp 00007ffe825ca1f0 error 6 in >> kamailio[562fcc17c000+4a2000] >> >> [Wed Sep 18 12:49:33 2019] kamailio[17416]: segfault at 0 ip >> 0000562fcc3544c0 sp 00007ffe825ca1f0 error 6 in >> kamailio[562fcc17c000+4a2000] >> >> [Wed Sep 18 12:49:33 2019] kamailio[17380]: segfault at 0 ip >> 00007f96c5dd099f sp 00007ffe825c98e0 error 4 in >> tm.so[7f96c5da5000+13f000] >> >> [Wed Sep 18 12:49:33 2019] kamailio[17381]: segfault at 0 ip >> 00007f96c5dd099f sp 00007ffe825c98e0 error 4 in >> tm.so[7f96c5da5000+13f000] >> >> [Wed Sep 18 12:49:34 2019] kamailio[17387]: segfault at 8 ip >> 0000562fcc25d1f3 sp 00007ffe825ca370 error 6 in >> kamailio[562fcc17c000+4a2000] >> >> >> and I was wondering if this core is related. >> >> >> Thanks >> >> >> On Fri, Sep 13, 2019 at 5:18 PM Alex Balashov >> <[email protected] <mailto:[email protected]>> wrote: >> >> Hi Daniel, >> >> Thanks a lot for this work; it's definitely important that >> Kamailio >> build against OpenSSL v1.1, and I'll be happy to contribute >> some testing >> in a live environment in the coming weeks. Much appreciated. >> >> Have a good weekend, >> >> -- Alex >> >> On Fri, Sep 13, 2019 at 01:34:12PM +0200, Daniel-Constantin >> Mierla wrote: >> >> > Hello, >> > after couple of months waiting for feedback about the >> workaround with the >> > preloaded shared library to set the PTHREAD_PROCESS_SHARED >> attribute for >> > pthread mutex/rwlock in order to avoid deadlocks and as it >> seems to be the >> > solution, I just pushed a few commits to move that code in >> the core of >> > kamailio. >> > >> > It this way the initialization with the >> PTHREAD_PROCESS_SHARED attribute is >> > done by default when libssl v1.1 is detected at compile >> time, without the >> > need of using the approach with a preloaded library. >> > >> > The shared library is still kept in the tls module: >> > * >> > >> >> https://github.com/kamailio/kamailio/tree/master/src/modules/tls/utils/openssl_mutex_shared >> > >> > Probably is going to be removed in the near future. >> > >> > I am asking the community members that use Kamailio on >> systems with libssl >> > v1.1+ to test the master branch and give feedback. The >> master branch is >> > anyhow in testing phase for preparing the next major >> release v5.3.0, so it >> > will help for that as well. >> > >> > If no issue is going to be reported soon, the commits will >> be backported to >> > stable branches, to re-enable smooth use of Kamailio with >> libssl v1.1+. >> > >> > When libssl v1.1+ is detected, the core links with pthread, >> but that was >> > already present in the Makefiles.defs for different compile >> options. The >> > pthread more or less a default library on linuxes/unixes, >> so this should >> > not introduce any major dependency to the Kamailio core. >> > >> > Cheers, >> > Daniel >> > >> > -- >> > Daniel-Constantin Mierla - http://www.asipto.com >> > http://twitter.com/#!/miconda - >> http://www.linkedin.com/in/miconda >> >> > _______________________________________________ >> > Kamailio (SER) - Users Mailing List >> > [email protected] >> <mailto:[email protected]> >> > https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users >> >> >> -- >> Alex Balashov | Principal | Evariste Systems LLC >> >> Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) >> Web: http://www.evaristesys.com/, http://www.csrpswitch.com/ >> >> _______________________________________________ >> Kamailio (SER) - Users Mailing List >> [email protected] <mailto:[email protected]> >> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users >> >> >> >> -- >> Andy Chen >> Sr. Telephony Lead Engineer >> 415 516 5535 (M) >> achen@ <mailto:[email protected]>fuze.com <http://fuze.com> >> >> >> *Confidentiality Notice: The information contained in this e-mail >> and any >> attachments may be confidential. If you are not an intended >> recipient, you >> are hereby notified that any dissemination, distribution or >> copying of this >> e-mail is strictly prohibited. If you have received this e-mail >> in error, >> please notify the sender and permanently delete the e-mail and any >> attachments immediately. You should not retain, copy or use this >> e-mail or >> any attachment for any purpose, nor disclose all or any part of the >> contents to any other person. Thank you.* >> >> _______________________________________________ >> Kamailio (SER) - Users Mailing List >> [email protected] <mailto:[email protected]> >> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users > > -- > Daniel-Constantin Mierla -- www.asipto.com <http://www.asipto.com> > www.twitter.com/miconda <http://www.twitter.com/miconda> -- > www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> > Kamailio Advanced Training, Oct 21-23, 2019, Berlin, Germany -- > https://asipto.com/u/kat > > > > -- > Andy Chen > Sr. Telephony Lead Engineer > 415 516 5535 (M) > achen@ <mailto:[email protected]>fuze.com <http://fuze.com> > > > *Confidentiality Notice: The information contained in this e-mail and any > attachments may be confidential. If you are not an intended recipient, you > are hereby notified that any dissemination, distribution or copying of > this > e-mail is strictly prohibited. If you have received this e-mail in error, > please notify the sender and permanently delete the e-mail and any > attachments immediately. You should not retain, copy or use this e-mail or > any attachment for any purpose, nor disclose all or any part of the > contents to any other person. Thank you.* -- Daniel-Constantin Mierla -- www.asipto.com www.twitter.com/miconda -- www.linkedin.com/in/miconda Kamailio Advanced Training, Oct 21-23, 2019, Berlin, Germany -- https://asipto.com/u/kat
_______________________________________________ Kamailio (SER) - Users Mailing List [email protected] https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
