[jira] [Updated] (IGNITE-15959) Remove operation fetches entry value into heap
[
https://issues.apache.org/jira/browse/IGNITE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15959:
--
Remaining Estimate: 40h (was: 168h)
Original Estimate: 40h (was: 168h)
> Remove operation fetches entry value into heap
> --
>
> Key: IGNITE-15959
> URL: https://issues.apache.org/jira/browse/IGNITE-15959
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 40h
> Remaining Estimate: 40h
>
> [See similar problem for the containsKey()
> operation|https://issues.apache.org/jira/browse/IGNITE-16137]
> {{IgniteCache#remove(key)}} operation fetches full entry into heap memory.
> This is inefficient when working with large objects: our application running
> with limited heap memory fails with {{java.lang.OutOfMemoryError: Java heap
> space}} when trying to remove an entry by key.
> It seems wrong that Ignite needs to fetch the full entry on heap to remove
> the entry. Please enhance Ignite to not be doing that or explain why Ignite
> must do that.
> h2. Reproducer
> h3. Steps
> Create a Gradle project with the below class and run it as
> {{./gradlew test --tests apache.ignite.issues.RemoveOperationHeapUsage}}
> {{build.gradle}}
> {code:groovy}
> test {
> minHeapSize = "512m"
> maxHeapSize = "512m"
> }
> {code}
> {{RemoveOperationHeapUsage.java}}
> {code:java}
> public class RemoveOperationHeapUsage {
> /** Run the test with -Xmx512m -Xms512m */
> @Test
> public void removeOperationFetchesValueOnHeap() {
> var igniteCfg = new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
> )
> .setCacheConfiguration(new CacheConfiguration<>("blobs"));
> try (var ignite = Ignition.start(igniteCfg)) {
> Cache cache = ignite.cache("blobs");
> // Put a BLOB having size of 35% of free memory to the cache
> Runtime.getRuntime().gc();
> var freeMemory = Runtime.getRuntime().freeMemory();
> var blobSize = (int)(freeMemory * 0.35);
> putBlob(cache, blobSize);
> // Use 70% of the free heap
> Runtime.getRuntime().gc();
> var unused = new byte[2 * blobSize];
> // Remove the blob from the cache.
> // This throws "OutOfMemoryError: Java heap space" since Ignite
> retrieves full entry to the heap.
> // Why does Ignite retrieve entry value to delete the entry?
> cache.remove(1);
> }
> }
> private static void putBlob(Cache cache, int blobSize) {
> var blob = new byte[blobSize];
> cache.put(1, blob);
> }
> }
> {code}
> h3. Expected
> The test passes
> h3. Actual
> The {{cache.remove}} operatoin fails with:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
> at
> org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.found(BPlusTree.java:3987)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:317)
> at
>
[jira] [Updated] (IGNITE-15959) Remove operation fetches entry value into heap
[
https://issues.apache.org/jira/browse/IGNITE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15959:
--
Remaining Estimate: 168h
Original Estimate: 168h
> Remove operation fetches entry value into heap
> --
>
> Key: IGNITE-15959
> URL: https://issues.apache.org/jira/browse/IGNITE-15959
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> [See similar problem for the containsKey()
> operation|https://issues.apache.org/jira/browse/IGNITE-16137]
> {{IgniteCache#remove(key)}} operation fetches full entry into heap memory.
> This is inefficient when working with large objects: our application running
> with limited heap memory fails with {{java.lang.OutOfMemoryError: Java heap
> space}} when trying to remove an entry by key.
> It seems wrong that Ignite needs to fetch the full entry on heap to remove
> the entry. Please enhance Ignite to not be doing that or explain why Ignite
> must do that.
> h2. Reproducer
> h3. Steps
> Create a Gradle project with the below class and run it as
> {{./gradlew test --tests apache.ignite.issues.RemoveOperationHeapUsage}}
> {{build.gradle}}
> {code:groovy}
> test {
> minHeapSize = "512m"
> maxHeapSize = "512m"
> }
> {code}
> {{RemoveOperationHeapUsage.java}}
> {code:java}
> public class RemoveOperationHeapUsage {
> /** Run the test with -Xmx512m -Xms512m */
> @Test
> public void removeOperationFetchesValueOnHeap() {
> var igniteCfg = new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
> )
> .setCacheConfiguration(new CacheConfiguration<>("blobs"));
> try (var ignite = Ignition.start(igniteCfg)) {
> Cache cache = ignite.cache("blobs");
> // Put a BLOB having size of 35% of free memory to the cache
> Runtime.getRuntime().gc();
> var freeMemory = Runtime.getRuntime().freeMemory();
> var blobSize = (int)(freeMemory * 0.35);
> putBlob(cache, blobSize);
> // Use 70% of the free heap
> Runtime.getRuntime().gc();
> var unused = new byte[2 * blobSize];
> // Remove the blob from the cache.
> // This throws "OutOfMemoryError: Java heap space" since Ignite
> retrieves full entry to the heap.
> // Why does Ignite retrieve entry value to delete the entry?
> cache.remove(1);
> }
> }
> private static void putBlob(Cache cache, int blobSize) {
> var blob = new byte[blobSize];
> cache.put(1, blob);
> }
> }
> {code}
> h3. Expected
> The test passes
> h3. Actual
> The {{cache.remove}} operatoin fails with:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
> at
> org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.found(BPlusTree.java:3987)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:317)
> at
>
[jira] [Updated] (IGNITE-15959) Remove operation fetches entry value into heap
[
https://issues.apache.org/jira/browse/IGNITE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15959:
--
Labels: cggg (was: )
> Remove operation fetches entry value into heap
> --
>
> Key: IGNITE-15959
> URL: https://issues.apache.org/jira/browse/IGNITE-15959
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.11
>Reporter: Alexey Kukushkin
>Priority: Major
> Labels: cggg
>
> [See similar problem for the containsKey()
> operation|https://issues.apache.org/jira/browse/IGNITE-16137]
> {{IgniteCache#remove(key)}} operation fetches full entry into heap memory.
> This is inefficient when working with large objects: our application running
> with limited heap memory fails with {{java.lang.OutOfMemoryError: Java heap
> space}} when trying to remove an entry by key.
> It seems wrong that Ignite needs to fetch the full entry on heap to remove
> the entry. Please enhance Ignite to not be doing that or explain why Ignite
> must do that.
> h2. Reproducer
> h3. Steps
> Create a Gradle project with the below class and run it as
> {{./gradlew test --tests apache.ignite.issues.RemoveOperationHeapUsage}}
> {{build.gradle}}
> {code:groovy}
> test {
> minHeapSize = "512m"
> maxHeapSize = "512m"
> }
> {code}
> {{RemoveOperationHeapUsage.java}}
> {code:java}
> public class RemoveOperationHeapUsage {
> /** Run the test with -Xmx512m -Xms512m */
> @Test
> public void removeOperationFetchesValueOnHeap() {
> var igniteCfg = new IgniteConfiguration()
> .setDiscoverySpi(
> new TcpDiscoverySpi()
> .setIpFinder(new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
> )
> .setCacheConfiguration(new CacheConfiguration<>("blobs"));
> try (var ignite = Ignition.start(igniteCfg)) {
> Cache cache = ignite.cache("blobs");
> // Put a BLOB having size of 35% of free memory to the cache
> Runtime.getRuntime().gc();
> var freeMemory = Runtime.getRuntime().freeMemory();
> var blobSize = (int)(freeMemory * 0.35);
> putBlob(cache, blobSize);
> // Use 70% of the free heap
> Runtime.getRuntime().gc();
> var unused = new byte[2 * blobSize];
> // Remove the blob from the cache.
> // This throws "OutOfMemoryError: Java heap space" since Ignite
> retrieves full entry to the heap.
> // Why does Ignite retrieve entry value to delete the entry?
> cache.remove(1);
> }
> }
> private static void putBlob(Cache cache, int blobSize) {
> var blob = new byte[blobSize];
> cache.put(1, blob);
> }
> }
> {code}
> h3. Expected
> The test passes
> h3. Actual
> The {{cache.remove}} operatoin fails with:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44)
> at
> org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
> at
> org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
> at
> org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
> at
> org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.found(BPlusTree.java:3987)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:317)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5921)
>
[jira] [Updated] (IGNITE-15959) Remove operation fetches entry value into heap
[
https://issues.apache.org/jira/browse/IGNITE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15959:
--
Description:
[See similar problem for the containsKey()
operation|https://issues.apache.org/jira/browse/IGNITE-16137]
{{IgniteCache#remove(key)}} operation fetches full entry into heap memory. This
is inefficient when working with large objects: our application running with
limited heap memory fails with {{java.lang.OutOfMemoryError: Java heap space}}
when trying to remove an entry by key.
It seems wrong that Ignite needs to fetch the full entry on heap to remove the
entry. Please enhance Ignite to not be doing that or explain why Ignite must do
that.
h2. Reproducer
h3. Steps
Create a Gradle project with the below class and run it as
{{./gradlew test --tests apache.ignite.issues.RemoveOperationHeapUsage}}
{{build.gradle}}
{code:groovy}
test {
minHeapSize = "512m"
maxHeapSize = "512m"
}
{code}
{{RemoveOperationHeapUsage.java}}
{code:java}
public class RemoveOperationHeapUsage {
/** Run the test with -Xmx512m -Xms512m */
@Test
public void removeOperationFetchesValueOnHeap() {
var igniteCfg = new IgniteConfiguration()
.setDiscoverySpi(
new TcpDiscoverySpi()
.setIpFinder(new
TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
)
.setCacheConfiguration(new CacheConfiguration<>("blobs"));
try (var ignite = Ignition.start(igniteCfg)) {
Cache cache = ignite.cache("blobs");
// Put a BLOB having size of 35% of free memory to the cache
Runtime.getRuntime().gc();
var freeMemory = Runtime.getRuntime().freeMemory();
var blobSize = (int)(freeMemory * 0.35);
putBlob(cache, blobSize);
// Use 70% of the free heap
Runtime.getRuntime().gc();
var unused = new byte[2 * blobSize];
// Remove the blob from the cache.
// This throws "OutOfMemoryError: Java heap space" since Ignite
retrieves full entry to the heap.
// Why does Ignite retrieve entry value to delete the entry?
cache.remove(1);
}
}
private static void putBlob(Cache cache, int blobSize) {
var blob = new byte[blobSize];
cache.put(1, blob);
}
}
{code}
h3. Expected
The test passes
h3. Actual
The {{cache.remove}} operatoin fails with:
{noformat}
java.lang.OutOfMemoryError: Java heap space
at
org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44)
at
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
at
org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.found(BPlusTree.java:3987)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:317)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5921)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:290)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5907)
at
org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:174)
at
org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:397)
at
[jira] [Updated] (IGNITE-15959) Remove operation fetches entry value into heap
[
https://issues.apache.org/jira/browse/IGNITE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15959:
--
Description:
{{IgniteCache#remove(key)}} operation fetches full entry into heap memory. This
is inefficient when working with large objects: our application running with
limited heap memory fails with {{java.lang.OutOfMemoryError: Java heap space}}
when trying to remove an entry by key.
It seems wrong that Ignite needs to fetch the full entry on heap to remove the
entry. Please enhance Ignite to not be doing that or explain why Ignite must do
that.
h2. Reproducer
h3. Steps
Create a Gradle project with the below class and run it as
{{./gradlew test --tests apache.ignite.issues.RemoveOperationHeapUsage}}
{{build.gradle}}
{code:groovy}
test {
minHeapSize = "512m"
maxHeapSize = "512m"
}
{code}
{{RemoveOperationHeapUsage.java}}
{code:java}
public class RemoveOperationHeapUsage {
/** Run the test with -Xmx512m -Xms512m */
@Test
public void removeOperationFetchesValueOnHeap() {
var igniteCfg = new IgniteConfiguration()
.setDiscoverySpi(
new TcpDiscoverySpi()
.setIpFinder(new
TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
)
.setCacheConfiguration(new CacheConfiguration<>("blobs"));
try (var ignite = Ignition.start(igniteCfg)) {
Cache cache = ignite.cache("blobs");
// Put a BLOB having size of 35% of free memory to the cache
Runtime.getRuntime().gc();
var freeMemory = Runtime.getRuntime().freeMemory();
var blobSize = (int)(freeMemory * 0.35);
putBlob(cache, blobSize);
// Use 70% of the free heap
Runtime.getRuntime().gc();
var unused = new byte[2 * blobSize];
// Remove the blob from the cache.
// This throws "OutOfMemoryError: Java heap space" since Ignite
retrieves full entry to the heap.
// Why does Ignite retrieve entry value to delete the entry?
cache.remove(1);
}
}
private static void putBlob(Cache cache, int blobSize) {
var blob = new byte[blobSize];
cache.put(1, blob);
}
}
{code}
h3. Expected
The test passes
h3. Actual
The {{cache.remove}} operatoin fails with:
{noformat}
java.lang.OutOfMemoryError: Java heap space
at
org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44)
at
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
at
org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.found(BPlusTree.java:3987)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:317)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5921)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:290)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5907)
at
org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:174)
at
org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:397)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.read(BPlusTree.java:6108)
at
[jira] [Updated] (IGNITE-15959) Remove operation fetches entry value into heap
[
https://issues.apache.org/jira/browse/IGNITE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15959:
--
Description:
{{IgniteCache#remove(key)}} operation fetches full entry into heap memory. This
makes Ignite inefficient with handling large objects: our application running
with limited heap memory fails with {{java.lang.OutOfMemoryError: Java heap
space}} when trying to remove an entry by key.
It seems wrong that Ignite needs to fetch the full entry on heap to remove the
entry. Please enhance Ignite to not be doing that or explain why Ignite must do
that.
h2. Reproducer
h3. Steps
Create a Gradle project with the below class and run it as
{{./gradlew test --tests apache.ignite.issues.RemoveOperationHeapUsage}}
{{build.gradle}}
{code:groovy}
test {
minHeapSize = "512m"
maxHeapSize = "512m"
}
{code}
{{RemoveOperationHeapUsage.java}}
{code:java}
public class RemoveOperationHeapUsage {
/** Run the test with -Xmx512m -Xms512m */
@Test
public void removeOperationFetchesValueOnHeap() {
var igniteCfg = new IgniteConfiguration()
.setDiscoverySpi(
new TcpDiscoverySpi()
.setIpFinder(new
TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
)
.setCacheConfiguration(new CacheConfiguration<>("blobs"));
try (var ignite = Ignition.start(igniteCfg)) {
Cache cache = ignite.cache("blobs");
// Put a BLOB having size of 35% of free memory to the cache
Runtime.getRuntime().gc();
var freeMemory = Runtime.getRuntime().freeMemory();
var blobSize = (int)(freeMemory * 0.35);
putBlob(cache, blobSize);
// Use 70% of the free heap
Runtime.getRuntime().gc();
var unused = new byte[2 * blobSize];
// Remove the blob from the cache.
// This throws OutOfMemoryError: Java heap space since Ignite
retrieves full entry to the heap.
// Why does Ignite retrieve entry value to delete the entry?
cache.remove(1);
}
}
private static void putBlob(Cache cache, int blobSize) {
var blob = new byte[blobSize];
cache.put(1, blob);
}
}
{code}
h3. Expected
The test passes
h3. Actual
The {{cache.remove}} operatoin fails with:
{noformat}
java.lang.OutOfMemoryError: Java heap space
at
org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44)
at
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
at
org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.found(BPlusTree.java:3987)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:317)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5921)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:290)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5907)
at
org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:174)
at
org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:397)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.read(BPlusTree.java:6108)
at
[jira] [Updated] (IGNITE-15959) Remove operation fetches entry value into heap
[
https://issues.apache.org/jira/browse/IGNITE-15959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Kukushkin updated IGNITE-15959:
--
Description:
{{IgniteCache#remove(key)}} operation fetches full entry into heap memory. This
makes Ignite inefficient with handling large objects: our application running
with limited heap memory fails with {{java.lang.OutOfMemoryError: Java heap
space}} when trying to remove an entry by key.
It seems wrong that Ignite needs to fetch the full entry on heap to remove the
entry. Please enhance Ignite to not be doing that or explain why Ignite must do
that.
h2. Reproducer
h3. Steps
Create a Gradle project with the below class and run it as
{{./gradlew test --tests apache.ignite.issues.RemoveOperationHeapUsage}}
{{build.gradle}}
{code:groovy}
test {
minHeapSize = "512m"
maxHeapSize = "512m"
}
{code}
{{RemoveOperationHeapUsage.java}}
{code:java}
public class RemoveOperationHeapUsage {
/** Run the test with -Xmx512m -Xms512m */
@Test
public void removeOperationFetchesValueOnHeap() {
var igniteCfg = new IgniteConfiguration()
.setDiscoverySpi(
new TcpDiscoverySpi()
.setIpFinder(new
TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500")))
)
.setCacheConfiguration(new CacheConfiguration<>("blobs"));
try (var ignite = Ignition.start(igniteCfg)) {
Cache cache = ignite.cache("blobs");
// Put a BLOB having size of 35% of free memory to the cache
Runtime.getRuntime().gc();
var freeMemory = Runtime.getRuntime().freeMemory();
var blobSize = (int)(freeMemory * 0.35);
putBlob(cache, blobSize);
// Use 80% of the free heap
Runtime.getRuntime().gc();
var unused = new byte[2 * blobSize];
// Remove the blob from the cache.
// This throws OutOfMemoryError: Java heap space since Ignite
retrieves full entry to the heap.
// Why does Ignite retrieve entry value to delete the entry?
cache.remove(1);
}
}
private static void putBlob(Cache cache, int blobSize) {
var blob = new byte[blobSize];
cache.put(1, blob);
}
}
{code}
h3. Expected
The test passes
h3. Actual
The {{cache.remove}} operatoin fails with:
{noformat}
java.lang.OutOfMemoryError: Java heap space
at
org.apache.ignite.internal.processors.cache.IncompleteCacheObject.(IncompleteCacheObject.java:44)
at
org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl.toCacheObject(CacheObjectBinaryProcessorImpl.java:1385)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncompleteValue(CacheDataRowAdapter.java:680)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readFragment(CacheDataRowAdapter.java:500)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.readIncomplete(CacheDataRowAdapter.java:411)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.doInitFromLink(CacheDataRowAdapter.java:316)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:165)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:136)
at
org.apache.ignite.internal.processors.cache.tree.DataRow.(DataRow.java:55)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataRowStore.dataRow(CacheDataRowStore.java:129)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:422)
at
org.apache.ignite.internal.processors.cache.tree.CacheDataTree.getRow(CacheDataTree.java:63)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Invoke.found(BPlusTree.java:3987)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:317)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5921)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:290)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5907)
at
org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:174)
at
org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:397)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.read(BPlusTree.java:6108)
at
