[
https://issues.apache.org/jira/browse/IGNITE-15245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17618700#comment-17618700
]
Yury Gerzhedovich commented on IGNITE-15245:
--------------------------------------------
[~xtern] , LGTM, thanks for the contribution!
> JDBC connection leak with cache.invoke() over cache store with external JDBC
> storage
> ------------------------------------------------------------------------------------
>
> Key: IGNITE-15245
> URL: https://issues.apache.org/jira/browse/IGNITE-15245
> Project: Ignite
> Issue Type: Bug
> Components: cache
> Affects Versions: 2.10
> Reporter: Ilya Korol
> Assignee: Pavel Pereslegin
> Priority: Major
> Fix For: 2.15
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Given following snippet:
> {code:java}
> try (Transaction tx =
> ignite.transactions().txStart(TransactionConcurrency.PESSIMISTIC,
> TransactionIsolation.REPEATABLE_READ)) {
> cache.invoke(pojo.getId(), entryProcessor, pojo);
> tx.commit();
> }
> {code}
> If we run this over the cache that uses external storage (e.g. mysql), we may
> get exceptions like:
> {code:java}
> org.apache.ignite.IgniteCheckedException: Failed to load object ...
> at
> org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.loadFromStore(GridCacheStoreManagerAdapter.java:334)
> at
> org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.load(GridCacheStoreManagerAdapter.java:292)
> at
> org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.loadAllFromStore(GridCacheStoreManagerAdapter.java:433)
> at
> org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.loadAll(GridCacheStoreManagerAdapter.java:399)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLockFuture.loadMissingFromStore(GridDhtLockFuture.java:1111)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLockFuture.onComplete(GridDhtLockFuture.java:790)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLockFuture.onDone(GridDhtLockFuture.java:758)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLockFuture.onDone(GridDhtLockFuture.java:91)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:475)
> at
> org.apache.ignite.internal.util.future.GridCompoundFuture.checkComplete(GridCompoundFuture.java:284)
> at
> org.apache.ignite.internal.util.future.GridCompoundFuture.markInitialized(GridCompoundFuture.java:273)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLockFuture.map(GridDhtLockFuture.java:1052)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLockFuture.onOwnerChanged(GridDhtLockFuture.java:709)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMvccManager.notifyOwnerChanged(GridCacheMvccManager.java:226)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMvccManager.access$200(GridCacheMvccManager.java:81)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMvccManager$3.onOwnerChanged(GridCacheMvccManager.java:163)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.checkOwnerChanged(GridCacheMapEntry.java:5043)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.checkOwnerChanged(GridCacheMapEntry.java:4995)
> at
> org.apache.ignite.internal.processors.cache.distributed.GridDistributedCacheEntry.readyLock(GridDistributedCacheEntry.java:515)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLockFuture.readyLocks(GridDhtLockFuture.java:617)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtLockFuture.map(GridDhtLockFuture.java:825)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.lockAllAsyncInternal(GridDhtTransactionalCacheAdapter.java:1027)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.obtainLockAsync(GridDhtTxLocalAdapter.java:720)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.lockAllAsync(GridDhtTxLocalAdapter.java:665)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.lockAllAsync(GridDhtTransactionalCacheAdapter.java:1238)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest0(GridDhtTransactionalCacheAdapter.java:823)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearLockRequest(GridDhtTransactionalCacheAdapter.java:801)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.access$000(GridDhtTransactionalCacheAdapter.java:113)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$3.apply(GridDhtTransactionalCacheAdapter.java:159)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$3.apply(GridDhtTransactionalCacheAdapter.java:157)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1142)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:591)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1719)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1326)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4600(GridIoManager.java:157)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$8.execute(GridIoManager.java:1211)
> at
> org.apache.ignite.internal.managers.communication.TraceRunnable.run(TraceRunnable.java:54)
> at
> org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:564)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: javax.cache.integration.CacheLoaderException: Failed to load
> object ...
> at
> org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.load(CacheAbstractJdbcStore.java:853)
> at
> org.apache.ignite.internal.processors.cache.store.GridCacheWriteBehindStore.load(GridCacheWriteBehindStore.java:530)
> at
> org.apache.ignite.internal.processors.cache.CacheStoreBalancingWrapper.load(CacheStoreBalancingWrapper.java:97)
> at
> org.apache.ignite.internal.processors.cache.store.GridCacheStoreManagerAdapter.loadFromStore(GridCacheStoreManagerAdapter.java:326)
> ... 43 more
> Caused by: java.sql.SQLTransientConnectionException: HikariPool-1 -
> Connection is not available, request timed out after 30008ms.
> at
> com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:695)
> at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:197)
> at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:162)
> at
> com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:128)
> at
> org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.openConnection(CacheAbstractJdbcStore.java:324)
> at
> org.apache.ignite.cache.store.jdbc.CacheAbstractJdbcStore.connection(CacheAbstractJdbcStore.java
> {code}
> This exception was caught when Hikari data source is used for pooling, but
> this case is also true if we use plain Mysql data source implementation.
> This happens because when transaction is commited Ignite opens JDBC
> connection to underlying storage but doesn't close it when there is a
> transaction. Check the code of
> GridCacheWriteBehindStore.closeConnection(connection):
> {code:java}
> protected void closeConnection(@Nullable Connection conn) {
> CacheStoreSession ses = session();
> // Close connection right away if there is no transaction.
> if (ses.transaction() == null)
> U.closeQuiet(conn);
> }
> {code}
> I guess that because connection release should be done somewhere else.
> GridCacheWriteBehindStore.sessionEnd() is quite obvious place for this
> purpose, but its 'noop' somewhy. I suspect that it is because sessionEnd()
> was declared as deprecated, but logic for closing cache session was lost
> during refactoring. It looks like we can fix this issue If we add a call to
> underlying cache store:
> {code:java}
> @Override public void sessionEnd(boolean commit) {
> --- // No-op
> +++ store.sessionEnd(commit);
> }
> {code}
> I've created a test to reproduce this issue, and looks like proposed fix do
> the job, but I guess that this solution may be insufficient because I do not
> have enough knowledge about involved components so I may miss something and
> also because sessionEnd() now considered as deprecated.
> [TEST/REPRODUCER|https://github.com/solveme/ignite/blob/ignite-15245/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/store/LeakedConnectionTest.java]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
