[
https://issues.apache.org/jira/browse/MESOS-10186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17209996#comment-17209996
]
Jaya Prasad Reddy G commented on MESOS-10186:
---------------------------------------------
[~bmahler] any update on this?
> Segmentation fault while running mesos in SSL mode
> --------------------------------------------------
>
> Key: MESOS-10186
> URL: https://issues.apache.org/jira/browse/MESOS-10186
> Project: Mesos
> Issue Type: Bug
> Affects Versions: 1.7.0, 1.9.0
> Reporter: Jaya Prasad Reddy G
> Priority: Blocker
>
> Hello,
> I've been runnning into segmentation faults while running mesos in SSL mode.
> After backtracing the coredump, this is what I found:
> {code:java}
> #0 0x00007f93060b7592 in free () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x00007f9303ebe8cd in CRYPTO_free () from
> /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
> #2 0x00007f9303f7bfde in EVP_CIPHER_CTX_cleanup () from
> /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
> #3 0x00007f93042e4275 in ?? () from /lib/x86_64-linux-gnu/libssl.so.1.0.0
> #4 0x00007f93042e59ca in SSL_set_accept_state () from
> /lib/x86_64-linux-gnu/libssl.so.1.0.0
> #5 0x00007f93059a5c72 in ?? () from
> /usr/lib/x86_64-linux-gnu/libevent_openssl-2.0.so.5
> #6 0x00007f930a801a74 in
> process::network::internal::LibeventSSLSocketImpl::accept_SSL_callback
> (request=request@entry=0x7f92f8000a80) at
> src/posix/libevent/libevent_ssl_socket.cpp:1172
> #7 0x00007f930a8021ea in
> process::network::internal::LibeventSSLSocketImpl::accept_callback
> (this=this@entry=0x55569161b430, request=request@entry=0x7f92f8000a80) at
> src/posix/libevent/libevent_ssl_socket.cpp:1124
> #8 0x00007f930a8027bc in
> process::network::internal::LibeventSSLSocketImpl::<lambda(evconnlistener*,
> int, sockaddr*, int, void*)>::operator()(evconnlistener *, int, sockaddr *,
> void *, int) (listener=0x555691667980, socket=24,
> addr=<optimized out>, arg=0x55569161be30, addr_length=<optimized out>,
> __closure=<optimized out>) at src/posix/libevent/libevent_ssl_socket.cpp:988
> #9 0x00007f9305e0829c in ?? () from
> /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5
> #10 0x00007f9305dfa639 in event_base_loop () from
> /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5
> #11 0x00007f930a81c41d in process::EventLoop::run () at
> src/posix/libevent/libevent.cpp:98
> #12 0x00007f93066cbd00 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> #13 0x00007f930699c6ba in start_thread () from
> /lib/x86_64-linux-gnu/libpthread.so.0
> #14 0x00007f930613a60d in clone () from /lib/x86_64-linux-gnu/libc.so.6
> {code}
> The environment variables exported before running mesos are:
> {code:java}
> MESOS_SSL_CIPHERS=ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA
>
> LIBPROCESS_SSL_ENABLED=true
> LIBPROCESS_SSL_SUPPORT_DOWNGRADE=false
> LIBPROCESS_SSL_CA_DIR=/opt/mesos/secrets/certs/
> LIBPROCESS_SSL_KEY_FILE=/opt/mesos/secrets/certs/mesos.private_key
> LIBPROCESS_SSL_CERT_FILE=/opt/mesos/secrets/certs/mesos.certificate
> LIBPROCESS_SSL_VERIFY_CERT=false
> LIBPROCESS_SSL_REQUIRE_CERT=false
> LIBPROCESS_SSL_VERIFY_IPADD=false
> LIBPROCESS_SSL_CA_FILE=/etc/ssl_ca/ca_list.pem
> LIBPROCESS_SSL_CIPHERS=ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA
>
> LIBPROCESS_SSL_ENABLE_SSL_V3=false
> LIBPROCESS_SSL_ENABLE_TLS_V1_0=false
> LIBPROCESS_SSL_ENABLE_TLS_V1_1=false
> LIBPROCESS_SSL_ENABLE_TLS_V1_2=true
> ZOOKEEPER_SSL_ENABLED=0
> ZOOKEEPER_SSL_VERIFY_CERT=false
> ZOOKEEPER_SSL_REQUIRE_CERT=false
> ZOOKEEPER_SSL_CA_FILE=/etc/ssl_ca/ca_list.pem
> ZOOKEEPER_SSL_KEY_FILE=/opt/mesos/secrets/certs/mesos.private_key
> ZOOKEEPER_SSL_CERT_FILE=/opt/mesos/secrets/certs/mesos.certificate
> {code}
> And the command used to run mesos:
> {code:java}
> mesos_master --ip=172.24.51.99 --advertise_ip=172.24.51.99
> --hostname_lookup=false --acls=file:///opt/mesos/etc/acl.json
> --modules=file:///opt/mesos/etc/modules.json --port=5050 --quorum=3
> --work_dir=/ghostcache2/mesos/registry
> --zk=zk://mycluster1.random.cluster.com:2181,mycluster2.random.cluster.com:2181,mycluster3.random.cluster.com:2181,mycluster4.random.cluster.com:2181,mycluster5.random.cluster.com:2181/mesos.mycluster
> --cluster=mycluster --hostname=mycluster2.random.cluster.com
> --offer_timeout=30secs --webui_dir=/opt/mesos/share/mesos/webui
> --whitelist=file:///opt/mesos/etc/whitelist --authenticate_agents=true
> --authenticators=crammd5 --authorizers=local
> --credentials=file:///opt/mesos/secrets/all_credentials.json
> --authenticate_frameworks=true{code}
> lddtree of the mesos_master
> {code:java}
> ~# lddtree /a/sbin/mesos_master
> mesos_master => /a/sbin/mesos_master (interpreter =>
> /lib64/ld-linux-x86-64.so.2)
> libmesos-1.9.0.so => /usr/local/myproject/opt/mesos/lib/libmesos-1.9.0.so
> libevent-2.0.so.5 => /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5
> libblkid.so.1 => /lib/x86_64-linux-gnu/libblkid.so.1
> libevent_openssl-2.0.so.5 =>
> /usr/lib/x86_64-linux-gnu/libevent_openssl-2.0.so.5
> libevent_core-2.0.so.5 =>
> /usr/lib/x86_64-linux-gnu/libevent_core-2.0.so.5
> libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0
> libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
> libevent_pthreads-2.0.so.5 =>
> /usr/lib/x86_64-linux-gnu/libevent_pthreads-2.0.so.5
> libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1
> libcrypto.so.111.0.0 =>
> /usr/local/myproject/lib/openssl/x86_64-linux-gnu/libcrypto.so.111.0.0
> libssl.so.111.0.0 =>
> /usr/local/myproject/lib/openssl/x86_64-linux-gnu/libssl.so.111.0.0
> libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
> librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1
> libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
> ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
> libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
> libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6{code}
>
> I tried updating mesos(from 1.7.0 to 1.9.0), libevent(2.0.21 to 2.0.22, 2.1.8
> and 2.1.12) to latest versions but still no luck. Not sure what I'm missing.
> Any help is appreciated.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
