[
https://issues.apache.org/jira/browse/IGNITE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15959:
--------------------------------------
Remaining Estimate: 40h (was: 168h)
Original Estimate: 40h (was: 168h)
> Remove operation fetches entry value into heap
> ----------------------------------------------
>
> Key: IGNITE-15959
> URL: https://issues.apache.org/jira/browse/IGNITE-15959
> Project: Ignite
> Issue Type: Improvement
> Affects Versions: 2.11
> Reporter: Alexey Kukushkin
> Priority: Major
> Labels: cggg
> Original Estimate: 40h
> Remaining Estimate: 40h
>
> [See similar problem for the containsKey()
> operation|https://issues.apache.org/jira/browse/IGNITE-16137]
> {{IgniteCache#remove(key)}} operation fetches full entry into heap memory.
> This is inefficient when working with large objects: our application running
> with limited heap memory fails with {{java.lang.OutOfMemoryError: Java heap
> space}} when trying to remove an entry by key.
> It seems wrong that Ignite needs to fetch the full entry on heap to remove
> the entry. Please enhance Ignite to not be doing that or explain why Ignite
> must do that.
> h2. Reproducer
> h3. Steps
> Create a Gradle project with the below class and run it as
> {{./gradlew test --tests apache.ignite.issues.RemoveOperationHeapUsage}}
> {{build.gradle}}
> {code:groovy}
> test {
> minHeapSize = "512m"
> maxHeapSize = "512m"
> }
> {code}
> {{RemoveOperationHeapUsage.java}}
> {code:java}
> public class RemoveOperationHeapUsage {
> /** Run the test with -Xmx512m -Xms512m */
> @Test
> public void removeOperationFetchesValueOnHeap() {
> var igniteCfg = new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
> )
> .setCacheConfiguration(new CacheConfiguration<>("blobs"));
> try (var ignite = Ignition.start(igniteCfg)) {
> Cache<Integer, byte[]> cache = ignite.cache("blobs");
> // Put a BLOB having size of 35% of free memory to the cache
> Runtime.getRuntime().gc();
> var freeMemory = Runtime.getRuntime().freeMemory();
> var blobSize = (int)(freeMemory * 0.35);
> putBlob(cache, blobSize);
> // Use 70% of the free heap
> Runtime.getRuntime().gc();
> var unused = new byte[2 * blobSize];
> // Remove the blob from the cache.
> // This throws "OutOfMemoryError: Java heap space" since Ignite
> retrieves full entry to the heap.
> // Why does Ignite retrieve entry value to delete the entry?
> cache.remove(1);
> }
> }
> private static void putBlob(Cache<Integer, byte[]> cache, int blobSize) {
> var blob = new byte[blobSize];
> cache.put(1, blob);
> }
> }
> {code}
> h3. Expected
> The test passes
> h3. Actual
> The {{cache.remove}} operatoin fails with:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.ignite.internal.processors.cache.IncompleteCacheObject.<init>(IncompleteCacheObject.java:44)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
> at
> org.apache.ignite.internal.processors.cache.tree.DataRow.<init>(DataRow.java:55)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.found(BPlusTree.java:3987)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:317)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5921)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:290)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5907)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:174)
> at
> org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:397)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.read(BPlusTree.java:6108)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invokeDown(BPlusTree.java:1991)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1920)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1765)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1748)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:441)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:2342)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2589)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:2049)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1866)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1725)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:306)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:487)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
