Thank you Tito,

It looks like a crash in the memory manager, while trying to allocate a new structure. Do debug such problems there is no other way than enabling debugging for the memory manager (QM_MALLOC + DBG_MALLOC flags) - is this a production system with considerable load on it ?

Regards,

Bogdan-Andrei Iancu
OpenSIPS Founder and Developer
http://www.opensips-solutions.com

On 01.07.2016 18:44, Tito Cumpen wrote:
Bogdan,


Correction I was using :

version: opensips 2.2.0 (x86_64/linux)

flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, F_MALLOC, FAST_LOCK-ADAPTIVE_WAIT

ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535

poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.

git revision: d835721

main.c compiled on 15:24:26 Jun 28 2016 with gcc 4.8.5





On Fri, Jul 1, 2016 at 9:23 AM, Tito Cumpen <[email protected] <mailto:[email protected]>> wrote:

    Bogdan,

    Here is the backtrace:


    Reading symbols from /usr/sbin/opensips...done.

    [New LWP 2648]

    [Thread debugging using libthread_db enabled]

    Using host libthread_db library "/lib64/libthread_db.so.1".

    Core was generated by `/sbin/opensips -P /var/run/opensips.pid -u
    opensips -g opensips -M 1024 -f /etc'.

    Program terminated with signal 11, Segmentation fault.

    #0  0x0000000000512782 in fm_remove_free (n=0x7fadc0898210,
    qm=0x7fadc0809010) at mem/f_malloc.c:187

    187*pf=n->u.nxt_free;

    Missing separate debuginfos, use: debuginfo-install
    cyrus-sasl-lib-2.1.26-20.el7_2.x86_64
    glibc-2.17-106.el7_2.4.x86_64 gmp-6.0.0-12.el7_1.x86_64
    gnutls-3.3.8-14.el7_2.x86_64 keyutils-libs-1.5.8-3.el7.x86_64
    krb5-libs-1.13.2-12.el7_2.x86_64 libcom_err-1.42.9-7.el7.x86_64
    libcurl-7.29.0-25.el7.centos.x86_64 libffi-3.0.13-16.el7.x86_64
    libgcc-4.8.5-4.el7.x86_64 libgcrypt-1.5.3-12.el7_1.1.x86_64
    libgpg-error-1.12-3.el7.x86_64 libidn-1.28-4.el7.x86_64
    libmicrohttpd-0.9.33-2.el7.x86_64 librabbitmq-0.5.2-1.el7.x86_64
    libselinux-2.2.2-6.el7.x86_64 libssh2-1.4.3-10.el7_2.1.x86_64
    libstdc++-4.8.5-4.el7.x86_64 libtasn1-3.8-2.el7.x86_64
    nettle-2.7.1-4.el7.x86_64 nspr-4.11.0-1.el7_2.x86_64
    nss-3.21.0-9.el7_2.x86_64
    nss-softokn-freebl-3.16.2.3-14.2.el7_2.x86_64
    nss-util-3.21.0-2.2.el7_2.x86_64 openldap-2.4.40-9.el7_2.x86_64
    openssl-libs-1.0.1e-51.el7_2.4.x86_64 p11-kit-0.20.7-3.el7.x86_64
    pcre-8.32-15.el7.x86_64 trousers-0.3.13-1.el7.x86_64
    xz-libs-5.1.2-12alpha.el7.x86_64 zlib-1.2.7-15.el7.x86_64

    (gdb) bt full

    #0  0x0000000000512782 in fm_remove_free (n=0x7fadc0898210,
    qm=0x7fadc0809010) at mem/f_malloc.c:187

            pf = 0x0

            hash = 2051

    #1  fm_malloc (qm=0x7fadc0809010, size=size@entry=65608) at
    mem/f_malloc.c:415

            frag = 0x7fadc0898210

            n = <optimized out>

            hash = 2051

            __FUNCTION__ = "fm_malloc"

    #2  0x00007fadb4615091 in ws_process (con=0x7fadbeb3f718) at
    ../proto_ws/ws_common.h:576

            size = <optimized out>

            req = <optimized out>

            ret_code = WS_ERR_NONE

            bk = <optimized out>

            msg_len = <optimized out>

            local_rcv = {src_ip = {af = 3082146304 <tel:3082146304>,
    len = 32685, u = {addrl = {34359801655, 140720703251544}, addr32 =
    {63287, 8, 394765400, 32764}, addr16 = {63287, 0, 8, 0, 42072,
    6023, 32764, 0},

                  addr =
    "7\367\000\000\b\000\000\000X\244\207\027\374\177\000"}}, dst_ip =
    {af = 3080002780, len = 32685, u = {addrl = {4, 63119970000},
    addr32 = {4, 0, 2990427856, 14}, addr16 = {4, 0, 0,

                    0, 20176, 45630, 14, 0}, addr =
    "\004\000\000\000\000\000\000\000\320N>\262\016\000\000"}},
    src_port = 52976, dst_port = 49287, proto = 32685, proto_reserved1
    = -1214962633,

              proto_reserved2 = 32685, src_su = {s = {sa_family = 2,
    sa_data = ")\350n\343\276C\360#%\275\016\000\000"}, sin =
    {sin_family = 2, sin_port = 59433, sin_addr = {s_addr = 1136583534},

                  sin_zero = "\360#%\275\016\000\000"}, sin6 =
    {sin6_family = 2, sin6_port = 59433, sin6_flowinfo = 1136583534,
    sin6_addr = {__in6_u = {

                      __u6_addr8 =
    "\360#%\275\016\000\000\000\360·\300\255\177\000", __u6_addr16 =
    {9200, 48421, 14, 0, 52976, 49287, 32685, 0}, __u6_addr32 =
    {3173327856, 14, 3230125808, 32685}}},

                  sin6_scope_id = 1}}, bind_address = 0x7fadc0882ea8}

            newreq = <optimized out>

            msg_buf = <optimized out>

    #3  wss_read_req (con=0x7fadbeb3f718, bytes_read=<optimized out>)
    at proto_wss.c:428

            size = <optimized out>

            __FUNCTION__ = "wss_read_req"

    #4  0x000000000059c374 in handle_io (fm=<optimized out>,
    idx=idx@entry=0, event_type=event_type@entry=1) at
    net/net_tcp_proc.c:205

            ret = 0

            n = <optimized out>

            con = 0x7fadbeb3f718

            s = 57

            rw = <optimized out>

            resp = <optimized out>

            response = {140384203757736, 1}

            __FUNCTION__ = "handle_io"

    #5  0x000000000059d914 in io_wait_loop_epoll (h=<optimized out>,
    t=<optimized out>, repeat=<optimized out>) at
    net/../io_wait_loop.h:221

            ret = <optimized out>

            e = <optimized out>

            n = 1

            r = 0

    #6  tcp_worker_proc (unix_sock=<optimized out>) at
    net/net_tcp_proc.c:312

            __FUNCTION__ = "tcp_worker_proc"

    #7  0x00000000005a7fc9 in tcp_start_processes
    (chd_rank=chd_rank@entry=0x8417e0 <chd_rank.11018>,
    startup_done=startup_done@entry=0x7fadbe978738) at net/net_tcp.c:1761

            r = 5

            reader_fd = {50, 52}

            pid = 0

            load_p = 0x7fadbe979398

            __FUNCTION__ = "tcp_start_processes"

    #8  0x0000000000419ef5 in main_loop () at main.c:677

            startup_done = 0x7fadbe978738

            chd_rank = 18

    #9  main (argc=<optimized out>, argv=<optimized out>) at main.c:1249

            cfg_stream = <optimized out>

            c = <optimized out>

            r = <optimized out>

            tmp = 0x7ffc1787bf65 ""

            tmp_len = <optimized out>

            port = <optimized out>

            proto = <optimized out>

            protos_no = <optimized out>

            options = 0x5dae60
    "f:cCm:M:b:l:n:N:rRvdDFETSVhw:t:u:g:P:G:W:o:"

    ---Type <return> to continue, or q <return> to quit---

            ret = -1

            seed = 995413301

            rfd = <optimized out>

            __FUNCTION__ = "main"



    currently using version: opensips 2.3.0-dev (x86_64/linux)

    flags: STATS: On, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC,
    QM_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT

    ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN
    16, MAX_URI_SIZE 1024, BUF_SIZE 65535

    poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.

    git revision: 92434ef

    main.c compiled on 15:05:06 Jun 28 2016 with gcc 4.8.5


    On Tue, Jun 28, 2016 at 8:26 AM, Bogdan-Andrei Iancu
    <[email protected] <mailto:[email protected]>> wrote:

        Hi Tito,

        If opensips crashes, were you able to extract a backtrace from
        the core file(s) ?

        Thanks and regards,

        Bogdan-Andrei Iancu
        OpenSIPS Founder and Developer
        http://www.opensips-solutions.com

        On 27.06.2016 21:20, Tito Cumpen wrote:
        Group,


        I am experiencing strange behavior when configuring sip tls
        on port 443. At time opensips crashes or stops accepting new
        connections. Here are the tcp configs I am using:

        #disable_tcp=no

        tcp_connection_lifetime=3600

        tcp_connect_timeout=3

        tcp_keepidle = 30

        tcp_keepinterval = 5

        tcp_keepalive = 1

        tcp_keepcount = 5

        tcp_max_msg_time = 8

        tcp_children=10


        Any idea what would case this? I am assuming there are probes
        out in the internet that eventually make opensips crash?

        Thanks,
        Tito


        _______________________________________________
        Users mailing list
        [email protected] <mailto:[email protected]>
        http://lists.opensips.org/cgi-bin/mailman/listinfo/users




_______________________________________________
Users mailing list
[email protected]
http://lists.opensips.org/cgi-bin/mailman/listinfo/users

Reply via email to