https://bz.apache.org/bugzilla/show_bug.cgi?id=57860
Bug ID: 57860
Summary: use mina with apr/tomcat native and it crashs
Product: Tomcat Native
Version: 1.1.33
Hardware: PC
OS: Linux
Status: NEW
Severity: critical
Priority: P2
Component: Library
Assignee: dev@tomcat.apache.org
Reporter: zhangwei_e...@163.com
Created attachment 32687
--> https://bz.apache.org/bugzilla/attachment.cgi?id=32687&action=edit
java crash log
I use mina with apr socket (AprSocketAcceptor and AprSocketConnector) for
better perfermonce.
the situation is simple:
1000 client connnet to mina server and keep alive connectons(long connection),
and another server tell the mina server to push group messages every one
minitue. After servel group push, the mina server not work and crashs.
i just download and compile apr-1.5.1.tar.gz and tomcat-native-1.1.33, and
modify the mina server code like this:
if(useAPR){
SimpleIoProcessorPool<AprSession> pool = new
SimpleIoProcessorPool<AprSession>(AprIoProcessor.class);
acceptor = new AprSocketAcceptor(executor, pool);
}else{
SimpleIoProcessorPool<NioSession> pool = new
SimpleIoProcessorPool<NioSession>(NioProcessor.class);
acceptor = new NioSocketAcceptor(executor, pool);
}
if(TcpServer.useAPR){
connector = new AprSocketConnector();
}else{
connector = new NioSocketConnector();
}
Stack: [0x00007f5a8d327000,0x00007f5a8d3a8000], sp=0x00007f5a8d3a6470, free
space=509k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [libtcnative-1.so.0.1.33+0x148cf] Java_org_apache_tomcat_jni_Poll_poll+0x7f
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J org.apache.tomcat.jni.Poll.poll(JJ[JZ)I
J org.apache.mina.transport.socket.apr.AprIoProcessor.select(J)I
j org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run()V+52
j org.apache.mina.util.NamePreservingRunnable.run()V+29
j
java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub
--------------- P R O C E S S ---------------
Use gdb to
[root@iZ25kxtgwarZ push_heartserver8888]# gdb /opt/jdk1.7.0_45/bin/java
core.17249
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-64.el6_5.2)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /opt/jdk1.7.0_45/bin/java...Missing separate debuginfo for
/opt/jdk1.7.0_45/bin/java
Try: yum --disablerepo='*' --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/b8/2a586842f6da3df8a61093daadf04180e85c16.debug
(no debugging symbols found)...done.
[New Thread 17292]
[New Thread 17312]
[New Thread 17303]
[New Thread 17294]
[New Thread 17252]
[New Thread 17307]
[New Thread 17313]
[New Thread 17304]
[New Thread 17260]
[New Thread 17255]
[New Thread 17320]
[New Thread 17251]
[New Thread 17277]
[New Thread 17325]
[New Thread 17265]
[New Thread 17249]
[New Thread 17285]
[New Thread 17295]
[New Thread 17344]
[New Thread 17343]
[New Thread 17266]
[New Thread 17256]
[New Thread 17250]
[New Thread 17290]
[New Thread 17349]
[New Thread 17253]
[New Thread 17263]
[New Thread 17262]
[New Thread 17346]
[New Thread 17258]
[New Thread 17254]
[New Thread 17272]
[New Thread 17267]
[New Thread 17347]
[New Thread 17342]
[New Thread 17287]
[New Thread 17268]
[New Thread 17257]
[New Thread 17271]
[New Thread 17286]
[New Thread 17273]
[New Thread 17297]
[New Thread 17278]
[New Thread 17274]
[New Thread 17264]
[New Thread 17348]
[New Thread 17261]
[New Thread 17259]
[New Thread 17288]
[New Thread 17308]
[New Thread 17282]
[New Thread 17345]
Missing separate debuginfo for /opt/jdk1.7.0_45/bin/../lib/amd64/jli/libjli.so
Try: yum --disablerepo='*' --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/34/5b529c72e59f506bf391be9a64cbbf7574211f
Missing separate debuginfo for /opt/jdk1.7.0_45/jre/lib/amd64/server/libjvm.so
Try: yum --disablerepo='*' --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/6f/31a11f5f50e47d14c9871057d41df781aa6ccd
Missing separate debuginfo for /opt/jdk1.7.0_45/jre/lib/amd64/libverify.so
Try: yum --disablerepo='*' --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/20/b03b77b651a0152ab8de47774328f0cef51987
Missing separate debuginfo for /opt/jdk1.7.0_45/jre/lib/amd64/libjava.so
Try: yum --disablerepo='*' --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/44/d3e7e98d5c21945a85d4ef6d5e6991a414ae6d
Missing separate debuginfo for /opt/jdk1.7.0_45/jre/lib/amd64/libzip.so
Try: yum --disablerepo='*' --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/07/c3e3ac6dc32953a6b623b6f9631d3ccc148800
Missing separate debuginfo for /opt/jdk1.7.0_45/jre/lib/amd64/libnio.so
Try: yum --disablerepo='*' --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/26/2c5a10aaf4e557c40616d0f8d2988fb9d0d323
Missing separate debuginfo for /opt/jdk1.7.0_45/jre/lib/amd64/libnet.so
Try: yum --disablerepo='*' --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/a5/ce95e2069f89c08f90a4876a18acf6a6fe28c2
Missing separate debuginfo for /usr/local/apr/lib/libtcnative-1.so.0.1.33
Try: yum --disablerepo='*' --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/64/c7c3ea9b1da0105a4831820cf77ce54b36dd5d
Missing separate debuginfo for /usr/local/apr/lib/libapr-1.so.0
Try: yum --disablerepo='*' --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/9f/82e698e6063202d19523bc3bbe9c96473fc73a
Missing separate debuginfo for
Try: yum --disablerepo='*' --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/1e/0a7d58f454926e2afb4797865d85801ed65ece
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols
found)...done.
[Thread debugging using libthread_db enabled]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /opt/jdk1.7.0_45/bin/../lib/amd64/jli/libjli.so...(no
debugging symbols found)...done.
Loaded symbols for /opt/jdk1.7.0_45/bin/../lib/amd64/jli/libjli.so
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols
found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /opt/jdk1.7.0_45/jre/lib/amd64/server/libjvm.so...Missing
separate debuginfo for /opt/jdk1.7.0_45/jre/lib/amd64/server/libjvm.so
Try: yum --disablerepo='*' --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/6f/31a11f5f50e47d14c9871057d41df781aa6ccd.debug
(no debugging symbols found)...done.
Loaded symbols for /opt/jdk1.7.0_45/jre/lib/amd64/server/libjvm.so
Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /opt/jdk1.7.0_45/jre/lib/amd64/libverify.so...Missing
separate debuginfo for /opt/jdk1.7.0_45/jre/lib/amd64/libverify.so
Try: yum --disablerepo='*' --enablerepo='*-debug*' install
/usr/lib/debug/.build-id/20/b03b77b651a0152ab8de47774328f0cef51987.debug
(no debugging symbols found)...done.
Loaded symbols for /opt/jdk1.7.0_45/jre/lib/amd64/libverify.so
Reading symbols from /opt/jdk1.7.0_45/jre/lib/amd64/libjava.so...(no debugging
symbols found)...done.
Loaded symbols for /opt/jdk1.7.0_45/jre/lib/amd64/libjava.so
Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libnss_files.so.2
Reading symbols from /opt/jdk1.7.0_45/jre/lib/amd64/libzip.so...(no debugging
symbols found)...done.
Loaded symbols for /opt/jdk1.7.0_45/jre/lib/amd64/libzip.so
Reading symbols from /opt/jdk1.7.0_45/jre/lib/amd64/libnio.so...(no debugging
symbols found)...done.
Loaded symbols for /opt/jdk1.7.0_45/jre/lib/amd64/libnio.so
Reading symbols from /opt/jdk1.7.0_45/jre/lib/amd64/libnet.so...(no debugging
symbols found)...done.
Loaded symbols for /opt/jdk1.7.0_45/jre/lib/amd64/libnet.so
Reading symbols from /usr/local/apr/lib/libtcnative-1.so.0.1.33...done.
Loaded symbols for /usr/local/apr/lib/libtcnative-1.so.0.1.33
Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib64/libssl.so.10
Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging symbols
found)...done.
Loaded symbols for /usr/lib64/libcrypto.so.10
Reading symbols from /usr/local/apr/lib/libapr-1.so.0...done.
Loaded symbols for /usr/local/apr/lib/libapr-1.so.0
Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libgssapi_krb5.so.2
Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib64/libkrb5.so.3
Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libk5crypto.so.3
Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libz.so.1
Reading symbols from /lib64/libfreebl3.so...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libfreebl3.so
Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libresolv.so.2...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libselinux.so.1...(no debugging symbols
found)...done.
Loaded symbols for /lib64/libselinux.so.1
Core was generated by `/opt/jdk1.7.0_45/bin/java -server -Xms14436m -Xms14436m
-Xss512k -XX:MaxDirectM'.
Program terminated with signal 6, Aborted.
#0 0x0000003712632925 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install
glibc-2.12-1.132.el6_5.2.x86_64 keyutils-libs-1.4-5.el6.x86_64
krb5-libs-1.10.3-33.el6.x86_64 libcom_err-1.41.12-21.el6.x86_64
libselinux-2.0.94-5.8.el6.x86_64 nss-softokn-freebl-3.14.3-10.el6_5.x86_64
openssl-1.0.1e-30.el6_6.7.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0 0x0000003712632925 in raise () from /lib64/libc.so.6
#1 0x0000003712634105 in abort () from /lib64/libc.so.6
#2 0x00007f5aef40f155 in os::abort(bool) () from
/opt/jdk1.7.0_45/jre/lib/amd64/server/libjvm.so
#3 0x00007f5aef58e087 in VMError::report_and_die() () from
/opt/jdk1.7.0_45/jre/lib/amd64/server/libjvm.so
#4 0x00007f5aef413adf in JVM_handle_linux_signal () from
/opt/jdk1.7.0_45/jre/lib/amd64/server/libjvm.so
#5 <signal handler called>
#6 Java_org_apache_tomcat_jni_Poll_poll (e=0x7f5a6801e1e8, o=<value optimized
out>, pollset=140028420158600, timeout=<value optimized out>,
set=0x7f5a8d3a64f0, remove=0 '\000') at src/poll.c:310
#7 0x00007f5ae517f6b5 in ?? ()
#8 0x00007f5a8d3a6530 in ?? ()
#9 0x0000000000000000 in ?? ()
(gdb)
the core file is too big(>500MB) to attach. if you wish, i can provide the
file.
and i tried to fix the null point error by adding checking code like this:
vim src/poll.c
TCN_IMPLEMENT_CALL(jint, Poll, poll)(TCN_STDARGS, jlong pollset,
jlong timeout, jlongArray set,
jboolean remove)
....
/* Find the minimum timeout */
APR_RING_FOREACH(ep, &p->poll_ring, tcn_pfde_t, link)
{
apr_interval_time_t socket_timeout = 0;
tcn_socket_t *s = (tcn_socket_t *)ep->fd.client_data;
if(s==NULL){
continue;
}
if (s->timeout == TCN_NO_SOCKET_TIMEOUT) {
socket_timeout = p->default_timeout;
}
else {
socket_timeout = s->timeout;
}
it then works and don't crash, but the mina can't receive messages any more
after some time(the time it should crashs and exits)
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
