Goodness Ayinmode created IGNITE-25719:
------------------------------------------

             Summary: Prevent OOM in GridDhtTxPrepareFuture and GridCacheMvcc
                 Key: IGNITE-25719
                 URL: https://issues.apache.org/jira/browse/IGNITE-25719
             Project: Ignite
          Issue Type: Improvement
          Components: cache
    Affects Versions: 2.16
            Reporter: Goodness Ayinmode


In large transactions involving thousands of keys, the methods 
[checkNeedRebalanceKeys()|https://github.com/apache/ignite/blob/2027c2ae1fe5bf516e1c6ec01764843e62b57a83/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java#L1142],
 
[forceRebalanceKeys()|https://github.com/apache/ignite/blob/2027c2ae1fe5bf516e1c6ec01764843e62b57a83/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java#L1171],
 and 
[readyLocks()|https://github.com/apache/ignite/blob/2027c2ae1fe5bf516e1c6ec01764843e62b57a83/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java#L662]
 within GridDhtTxPrepareFuture can cause excessive memory usage and potential 
OOM. These methods are called inside the 
[prepare()|https://github.com/apache/ignite/blob/2027c2ae1fe5bf516e1c6ec01764843e62b57a83/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/GridDhtTxPrepareFuture.java#L1077]
 where the number of keys they handle grows proportionally with the transaction 
size and remain in memory throughout. 

I also think that there is a risk of indefinite memory growth with 
[locs|https://github.com/apache/ignite/blob/e457dcc98e53cf131a90db78d1d5814a1e3a565b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheMvcc.java#L273]
 inside 
[GridCacheMvcc|https://github.com/apache/ignite/blob/2027c2ae1fe5bf516e1c6ec01764843e62b57a83/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheMvcc.java#L50],
 which holds local lock candidates. Methods like 
[add0()|https://github.com/apache/ignite/blob/e457dcc98e53cf131a90db78d1d5814a1e3a565b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheMvcc.java#L267],
 
[candidates|https://github.com/apache/ignite/blob/e457dcc98e53cf131a90db78d1d5814a1e3a565b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheMvcc.java#L1408],
 and [localOwners() 
|https://github.com/apache/ignite/blob/e457dcc98e53cf131a90db78d1d5814a1e3a565b/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheMvcc.java#L142]operate
 over locs. Under failure scenarios like timeouts or transaction aborts locs, 
cleanup logic (e.g. 
[releaseLocal(|https://github.com/apache/ignite/blob/2027c2ae1fe5bf516e1c6ec01764843e62b57a83/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheMvcc.java#L1200]))
 may be skipped or delayed. This can result in stale entries remaining in 
{{{}locs{}}}, causing it to grow indefinitely and increasing the risk of memory 
pressure or OOM.

Would it be possible and beneficial to introduce batching or timeout-based 
cleanup to mitigate these potential memory issues?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to