Hi Chalitha, Caused by: org.apache.tomcat.jdbc.pool.PoolExhaustedException: [pool-216-thread-7] Timeout: Pool empty. Unable to fetch a connection in 60 seconds, none available[size:50; busy:50; idle:0; lastwait:60000].
>From the above logs, the problem lies with the database connection which has pool size of 50. So it should be carbon db, because for other dbsm you are having 100 as the pool size. You need to increase it as well. On Thu, Jun 30, 2016 at 1:46 PM, Chalitha Waldeniyage <[email protected]> wrote: > Hi Danesh, > > Yes, We have done that on Registry,AM,UM,Conf,Gov databases up to 100 > which are pointed to oracle database. But for the carbon db we didn't > change it since it's local h2. > Do we need to change it also? > > On Thu, Jun 30, 2016 at 1:34 PM, Danesh Kuruppu <[email protected]> wrote: > >> Hi Chalitha, >> >> Can you increase the maxActive in Registry DB configuration. Seems like >> there are no available connections in tomcat jdbc pool. Please set this to >> 80 at least. >> >> Thanks >> Danesh >> >> On Thu, Jun 30, 2016 at 8:29 AM, Chalitha Waldeniyage <[email protected]> >> wrote: >> >>> Hi All, >>> >>> We have setup an API Manager cluster (beta 15th pack) fronted by nginx >>> with following details. >>> >>> Publisher >>> Two store nodes >>> Gateway Cluster(one manager with two workers) >>> IS as key manager(two node cluster) >>> DB-Oracle 12c >>> Userstore-Read/write ldap >>> >>> We have ran a long running test for some days and observed out of memory >>> on one of the IS nodes. Once we have cleaned all created heap dumps,logs >>> and try to restart the cluster again and found the following error >>> continuously on publisher, gateway and IS nodes. >>> >>> I have attached the current database running process details with [1]. >>> Further we have used the [2] for fine tunning the servers. >>> >>> Appreciate your feedback on resolving this. >>> >>> >>> >>> ERROR - TransactionManager Failed to start new registry transaction. >>> org.apache.tomcat.jdbc.pool.PoolExhaustedException: [pool-216-thread-7] >>> Timeout: Pool empty. Unable to fetch a connection in 60 seconds, none >>> available[size:50; busy:50; idle:0; lastwait:60000]. >>> at >>> org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:674) >>> at >>> org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:188) >>> at >>> org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:128) >>> at >>> org.wso2.carbon.registry.core.jdbc.dataaccess.JDBCTransactionManager.beginTransaction(JDBCTransactionManager.java:80) >>> at >>> org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.beginTransaction(EmbeddedRegistry.java:443) >>> at >>> org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.getTags(EmbeddedRegistry.java:1532) >>> at >>> org.wso2.carbon.registry.core.caching.CacheBackedRegistry.getTags(CacheBackedRegistry.java:403) >>> at >>> org.wso2.carbon.registry.core.session.UserRegistry.getTagsInternal(UserRegistry.java:1398) >>> at >>> org.wso2.carbon.registry.core.session.UserRegistry.access$2500(UserRegistry.java:60) >>> at >>> org.wso2.carbon.registry.core.session.UserRegistry$26.run(UserRegistry.java:1381) >>> at >>> org.wso2.carbon.registry.core.session.UserRegistry$26.run(UserRegistry.java:1378) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at >>> org.wso2.carbon.registry.core.session.UserRegistry.getTags(UserRegistry.java:1378) >>> at >>> org.wso2.carbon.registry.indexing.indexer.IndexDocumentCreator.addTags(IndexDocumentCreator.java:201) >>> at >>> org.wso2.carbon.registry.indexing.indexer.IndexDocumentCreator.createIndexDocument(IndexDocumentCreator.java:117) >>> at >>> org.wso2.carbon.registry.indexing.AsyncIndexer$IndexingTask.createIndexDocument(AsyncIndexer.java:212) >>> at >>> org.wso2.carbon.registry.indexing.AsyncIndexer$IndexingTask.run(AsyncIndexer.java:196) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>> at java.lang.Thread.run(Thread.java:745) >>> [2016-06-30 06:28:48,855] ERROR - AsyncIndexer Error while indexing. >>> org.wso2.carbon.registry.core.exceptions.RegistryException: Failed to >>> start new registry transaction. >>> at >>> org.wso2.carbon.registry.core.jdbc.dataaccess.JDBCTransactionManager.beginTransaction(JDBCTransactionManager.java:97) >>> at >>> org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.beginTransaction(EmbeddedRegistry.java:443) >>> at >>> org.wso2.carbon.registry.core.jdbc.EmbeddedRegistry.getTags(EmbeddedRegistry.java:1532) >>> at >>> org.wso2.carbon.registry.core.caching.CacheBackedRegistry.getTags(CacheBackedRegistry.java:403) >>> at >>> org.wso2.carbon.registry.core.session.UserRegistry.getTagsInternal(UserRegistry.java:1398) >>> at >>> org.wso2.carbon.registry.core.session.UserRegistry.access$2500(UserRegistry.java:60) >>> at >>> org.wso2.carbon.registry.core.session.UserRegistry$26.run(UserRegistry.java:1381) >>> at >>> org.wso2.carbon.registry.core.session.UserRegistry$26.run(UserRegistry.java:1378) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at >>> org.wso2.carbon.registry.core.session.UserRegistry.getTags(UserRegistry.java:1378) >>> at >>> org.wso2.carbon.registry.indexing.indexer.IndexDocumentCreator.addTags(IndexDocumentCreator.java:201) >>> at >>> org.wso2.carbon.registry.indexing.indexer.IndexDocumentCreator.createIndexDocument(IndexDocumentCreator.java:117) >>> at >>> org.wso2.carbon.registry.indexing.AsyncIndexer$IndexingTask.createIndexDocument(AsyncIndexer.java:212) >>> at >>> org.wso2.carbon.registry.indexing.AsyncIndexer$IndexingTask.run(AsyncIndexer.java:196) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>> at java.lang.Thread.run(Thread.java:745) >>> Caused by: org.apache.tomcat.jdbc.pool.PoolExhaustedException: >>> [pool-216-thread-7] Timeout: Pool empty. Unable to fetch a connection in 60 >>> seconds, none available[size:50; busy:50; idle:0; lastwait:60000]. >>> at >>> org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:674) >>> at >>> org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:188) >>> at >>> org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:128) >>> at >>> org.wso2.carbon.registry.core.jdbc.dataaccess.JDBCTransactionManager.beginTransaction(JDBCTransactionManager.java:80) >>> ... 18 more >>> >>> >>> [1] >>> >>> >>> >>> >>> >>> >>> [2] https://docs.wso2.com/display/AM1100/Tuning+Performance >>> >>> >>> Thank you, >>> -- >>> *Chalitha Maheshwari* >>> Software Engineer-QA, >>> WSO2 Inc. >>> >>> *E-mail:* [email protected] >>> *Mobile: *+94710 411 112 >>> >>> >>> >>> _______________________________________________ >>> Dev mailing list >>> [email protected] >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> >> -- >> >> Danesh Kuruppu >> Software Engineer >> WSO2 Inc, >> Mobile: +94 (77) 1690552 >> > > > > -- > *Chalitha Maheshwari* > Software Engineer-QA, > WSO2 Inc. > > *E-mail:* [email protected] > *Mobile: *+94710 411 112 > > _______________________________________________ > Dev mailing list > [email protected] > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- Thanks Abimaran Kugathasan Senior Software Engineer Email : [email protected] Mobile : +94 773922820 <http://stackoverflow.com/users/515034> <http://lk.linkedin.com/in/abimaran> <http://www.lkabimaran.blogspot.com/> <https://github.com/abimarank> <https://twitter.com/abimaran>
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
