Alex Rudyy created QPID-8080: -------------------------------- Summary: [Broker-J] Broker can crash when JDBC message store opens to many SQL connections to the RDBMS exceeding the maximum connection limit Key: QPID-8080 URL: https://issues.apache.org/jira/browse/QPID-8080 Project: Qpid Issue Type: Bug Components: Broker-J Affects Versions: qpid-java-6.1.5, qpid-java-broker-7.0.0, qpid-java-6.0.8 Environment: Broker-J with JDVC message store can crash when messaging client(s) consume or publish small messages in non-transactional manner (for example, when AUTO_ACK acknowledge mode is used by JMS client).
The broker creates \{{AsyncAutoCommitTransaction}} per each consumed/published message. Each transaction opens a separate connection to RDBMS server. The number of concurrent \{{AsyncAutoCommitTransaction}} can exceed the maximum connection limit. As result, sql connection establishment fails causing broker to crash with the stack trace like the one below: {noformat} ######################################################################## # # Unhandled Exception org.apache.qpid.server.util.ServerScopedRuntimeException: org.apache.qpid.server.store.StoreException: java.sql.SQLNonTransientConnectionException: Too many connections in Thread IO-/127.0.0.1:55228 # # Exiting # ######################################################################## org.apache.qpid.server.util.ServerScopedRuntimeException: org.apache.qpid.server.store.StoreException: java.sql.SQLNonTransientConnectionException: Too many connections at org.apache.qpid.server.protocol.v0_10.AMQPConnection_0_10Impl.lambda$received$1(AMQPConnection_0_10Impl.java:157) at java.security.AccessController.doPrivileged(Native Method) at org.apache.qpid.server.protocol.v0_10.AMQPConnection_0_10Impl.received(AMQPConnection_0_10Impl.java:141) at org.apache.qpid.server.transport.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:134) at org.apache.qpid.server.transport.NonBlockingConnection.processAmqpData(NonBlockingConnection.java:610) at org.apache.qpid.server.transport.NonBlockingConnectionPlainDelegate.processData(NonBlockingConnectionPlainDelegate.java:58) at org.apache.qpid.server.transport.NonBlockingConnection.doRead(NonBlockingConnection.java:496) at org.apache.qpid.server.transport.NonBlockingConnection.doWork(NonBlockingConnection.java:270) at org.apache.qpid.server.transport.NetworkConnectionScheduler.processConnection(NetworkConnectionScheduler.java:134) at org.apache.qpid.server.transport.SelectorThread$ConnectionProcessor.processConnection(SelectorThread.java:575) at org.apache.qpid.server.transport.SelectorThread$SelectionTask.performSelect(SelectorThread.java:366) at org.apache.qpid.server.transport.SelectorThread$SelectionTask.run(SelectorThread.java:97) at org.apache.qpid.server.transport.SelectorThread.run(SelectorThread.java:533) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.qpid.server.bytebuffer.QpidByteBufferFactory.lambda$null$0(QpidByteBufferFactory.java:464) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.qpid.server.store.StoreException: java.sql.SQLNonTransientConnectionException: Too many connections at org.apache.qpid.server.store.jdbc.AbstractJDBCMessageStore$JDBCTransaction.<init>(AbstractJDBCMessageStore.java:1065) at org.apache.qpid.server.store.jdbc.GenericAbstractJDBCMessageStore$RecordedJDBCTransaction.<init>(GenericAbstractJDBCMessageStore.java:122) at org.apache.qpid.server.store.jdbc.GenericAbstractJDBCMessageStore$RecordedJDBCTransaction.<init>(GenericAbstractJDBCMessageStore.java:118) at org.apache.qpid.server.store.jdbc.GenericAbstractJDBCMessageStore.newTransaction(GenericAbstractJDBCMessageStore.java:114) at org.apache.qpid.server.txn.AsyncAutoCommitTransaction.dequeue(AsyncAutoCommitTransaction.java:104) at org.apache.qpid.server.protocol.v0_10.ServerSession.acknowledge(ServerSession.java:1185) at org.apache.qpid.server.protocol.v0_10.ConsumerTarget_0_10.acknowledge(ConsumerTarget_0_10.java:401) at org.apache.qpid.server.protocol.v0_10.ExplicitAcceptDispositionChangeListener.onAccept(ExplicitAcceptDispositionChangeListener.java:51) at org.apache.qpid.server.protocol.v0_10.ServerSession$2.performAction(ServerSession.java:1000) at org.apache.qpid.server.protocol.v0_10.ServerSession.dispositionChange(ServerSession.java:1091) at org.apache.qpid.server.protocol.v0_10.ServerSession.accept(ServerSession.java:995) at org.apache.qpid.server.protocol.v0_10.ServerSessionDelegate.messageAccept(ServerSessionDelegate.java:142) at org.apache.qpid.server.protocol.v0_10.ServerSessionDelegate.messageAccept(ServerSessionDelegate.java:91) at org.apache.qpid.server.protocol.v0_10.transport.MessageAccept.dispatch(MessageAccept.java:90) at org.apache.qpid.server.protocol.v0_10.ServerSessionDelegate.command(ServerSessionDelegate.java:1832) at org.apache.qpid.server.protocol.v0_10.ServerSessionDelegate.command(ServerSessionDelegate.java:108) at org.apache.qpid.server.protocol.v0_10.ServerSessionDelegate.command(ServerSessionDelegate.java:91) at org.apache.qpid.server.protocol.v0_10.transport.Method.delegate(Method.java:153) at org.apache.qpid.server.protocol.v0_10.ServerSession.received(ServerSession.java:508) at org.apache.qpid.server.protocol.v0_10.ServerConnection.dispatch(ServerConnection.java:539) at org.apache.qpid.server.protocol.v0_10.ServerConnectionDelegate.handle(ServerConnectionDelegate.java:115) at org.apache.qpid.server.protocol.v0_10.ServerConnectionDelegate.handle(ServerConnectionDelegate.java:52) at org.apache.qpid.server.protocol.v0_10.transport.MethodDelegate.messageAccept(MethodDelegate.java:117) at org.apache.qpid.server.protocol.v0_10.transport.MessageAccept.dispatch(MessageAccept.java:90) at org.apache.qpid.server.protocol.v0_10.ServerConnectionDelegate.command(ServerConnectionDelegate.java:103) at org.apache.qpid.server.protocol.v0_10.ServerConnectionDelegate.command(ServerConnectionDelegate.java:52) at org.apache.qpid.server.protocol.v0_10.transport.Method.delegate(Method.java:153) at org.apache.qpid.server.protocol.v0_10.ServerConnection.received(ServerConnection.java:279) at org.apache.qpid.server.protocol.v0_10.ServerAssembler.emit(ServerAssembler.java:178) at org.apache.qpid.server.protocol.v0_10.ServerAssembler.assemble(ServerAssembler.java:271) at org.apache.qpid.server.protocol.v0_10.ServerAssembler.frame(ServerAssembler.java:205) at org.apache.qpid.server.protocol.v0_10.ServerAssembler.received(ServerAssembler.java:135) at org.apache.qpid.server.protocol.v0_10.ServerAssembler.lambda$received$0(ServerAssembler.java:96) at java.security.AccessController.doPrivileged(Native Method) at org.apache.qpid.server.protocol.v0_10.ServerAssembler.received(ServerAssembler.java:90) at org.apache.qpid.server.protocol.v0_10.ServerInputHandler.received(ServerInputHandler.java:184) at org.apache.qpid.server.protocol.v0_10.AMQPConnection_0_10Impl.lambda$received$1(AMQPConnection_0_10Impl.java:146) ... 16 more Caused by: java.sql.SQLNonTransientConnectionException: Too many connections at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.get(ExceptionMapper.java:175) at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.getException(ExceptionMapper.java:110) at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:1088) at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:493) at org.mariadb.jdbc.MariaDbConnection.newConnection(MariaDbConnection.java:150) at org.mariadb.jdbc.Driver.connect(Driver.java:86) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at org.apache.qpid.server.store.jdbc.DefaultConnectionProvider.getConnection(DefaultConnectionProvider.java:49) at org.apache.qpid.server.store.jdbc.GenericJDBCMessageStore.getConnection(GenericJDBCMessageStore.java:127) at org.apache.qpid.server.store.jdbc.AbstractJDBCMessageStore.newConnection(AbstractJDBCMessageStore.java:497) at org.apache.qpid.server.store.jdbc.AbstractJDBCMessageStore$JDBCTransaction.<init>(AbstractJDBCMessageStore.java:1061) ... 52 more Caused by: java.sql.SQLException: Too many connections at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.authentication(AbstractConnectProtocol.java:849) at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.handleConnectionPhases(AbstractConnectProtocol.java:775) at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(AbstractConnectProtocol.java:453) at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:1084) ... 61 more Process finished with exit code 1 {noformat} Only JDBC message stores with default connection provider (no pool) are affected by the issue. It can be work around by using connection pool, for example, BoneCP. Reporter: Alex Rudyy -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org