[jira] [Commented] (IGNITE-10684) Memory leak in persistent IgniteSet
[
https://issues.apache.org/jira/browse/IGNITE-10684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722856#comment-16722856
]
Alexey Belov commented on IGNITE-10684:
---
Looks like proper destroy of sets solves this issue. Here is my unit test:
{code:java}
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteSet;
import org.apache.ignite.Ignition;
import org.apache.ignite.cache.CacheMode;
import org.apache.ignite.configuration.*;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.UUID;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
/**
* @author Alexey Belov
*/
public class IgniteManagerTest {
protected final Logger log =
LoggerFactory.getLogger(IgniteManagerTest.class);
private ThreadPoolExecutor ex = new ThreadPoolExecutor(8, 8,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue());
@Test
public void start() throws Exception {
final IgniteConfiguration cfg = new IgniteConfiguration();
final DataStorageConfiguration dataStorageConfiguration = new
DataStorageConfiguration();
final String igniteStorageDir = "c:\\work\\garbage\\igniteTest\\" +
UUID.randomUUID().toString();
System.out.println(igniteStorageDir);
dataStorageConfiguration.setStoragePath(igniteStorageDir);
final DataRegionConfiguration defaultDataRegionConfiguration =
dataStorageConfiguration
.getDefaultDataRegionConfiguration();
defaultDataRegionConfiguration.setEvictionThreshold(0.9);
defaultDataRegionConfiguration.setMetricsEnabled(true);
defaultDataRegionConfiguration.setPersistenceEnabled(true);
dataStorageConfiguration.setWalMode(WALMode.NONE);
cfg.setDataStorageConfiguration(dataStorageConfiguration);
final Ignite ignite = Ignition.start(cfg);
ignite.cluster().active(true);
while (true) {
if (ex.getQueue().size() < 8) {
System.out.println("added task " + ex.getQueue().size() + " " +
ex.getActiveCount());
ex.execute(() -> runQueues(ignite));
}
Thread.sleep(1000);
}
}
private void runQueues(Ignite ignite) {
for (int j = 0; j < 10; j++) {
final CollectionConfiguration setConfig = new
CollectionConfiguration();
setConfig.setCacheMode(CacheMode.REPLICATED);
setConfig.setBackups(1);
final String name = "set-" + j + UUID.randomUUID().toString();
setConfig.setGroupName(name);
final IgniteSet set = ignite
.set(name, setConfig);
final int i1 = 1000;
for (int i = 0; i < i1; i++) {
final String elem1 = UUID.randomUUID().toString();
set.add(elem1);
}
log.info(j + "write");
set.clear();
set.close();
ignite.destroyCache(name);
ignite.destroyCache("datastructures_ATOMIC_REPLICATED_1@" + name);
ignite.destroyCache("ignite-sys-atomic-cache@" + name);
}
log.info("Finish!");
}
}
{code}
See the memory graph in attachment !IgniteManagerTest [4] - JProfiler 9.2.1
4.jpg!
> Memory leak in persistent IgniteSet
> ---
>
> Key: IGNITE-10684
> URL: https://issues.apache.org/jira/browse/IGNITE-10684
> Project: Ignite
> Issue Type: Bug
> Components: data structures, persistence
>Affects Versions: 2.7
>Reporter: Alexey Belov
>Priority: Critical
> Labels: Ignite, igniteset, memory-leak, persistant
> Fix For: None
>
> Attachments: IgniteManagerTest [4] - JProfiler 9.2.1 4.jpg,
> IgniteManagerTest.start - JProfiler 9.2.1 2.jpg, IgniteManagerTest.start -
> JProfiler 9.2.1 3.jpg, IgniteManagerTest.start - JProfiler 9.2.1.jpg
>
>
> Hello. I have found a leak in IgniteSet with using persistence. Here is my
> Unit Test:
> {code:java}
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteSet;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.cache.CacheMode;
> import org.apache.ignite.configuration.*;
> import org.junit.Test;
> import org.slf4j.Logger;
> import org.slf4j.LoggerFactory;
> import java.util.UUID;
> import java.util.concurrent.LinkedBlockingQueue;
> import java.util.concurrent.ThreadPoolExecutor;
> import java.util.concurrent.TimeUnit;
> /**
> * @author Alexey Belov
> */
> public class IgniteManagerTest {
> protected final Logger log =
> LoggerFactory.getLogger(IgniteManagerTest.class);
> private ThreadPoolExecutor ex = new ThreadPoolExecutor(8, 8,
> 0L, TimeUnit.MILLISECONDS,
> new
[jira] [Updated] (IGNITE-10684) Memory leak in persistent IgniteSet
[
https://issues.apache.org/jira/browse/IGNITE-10684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Belov updated IGNITE-10684:
--
Attachment: IgniteManagerTest [4] - JProfiler 9.2.1 4.jpg
> Memory leak in persistent IgniteSet
> ---
>
> Key: IGNITE-10684
> URL: https://issues.apache.org/jira/browse/IGNITE-10684
> Project: Ignite
> Issue Type: Bug
> Components: data structures, persistence
>Affects Versions: 2.7
>Reporter: Alexey Belov
>Priority: Critical
> Labels: Ignite, igniteset, memory-leak, persistant
> Fix For: None
>
> Attachments: IgniteManagerTest [4] - JProfiler 9.2.1 4.jpg,
> IgniteManagerTest.start - JProfiler 9.2.1 2.jpg, IgniteManagerTest.start -
> JProfiler 9.2.1 3.jpg, IgniteManagerTest.start - JProfiler 9.2.1.jpg
>
>
> Hello. I have found a leak in IgniteSet with using persistence. Here is my
> Unit Test:
> {code:java}
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteSet;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.cache.CacheMode;
> import org.apache.ignite.configuration.*;
> import org.junit.Test;
> import org.slf4j.Logger;
> import org.slf4j.LoggerFactory;
> import java.util.UUID;
> import java.util.concurrent.LinkedBlockingQueue;
> import java.util.concurrent.ThreadPoolExecutor;
> import java.util.concurrent.TimeUnit;
> /**
> * @author Alexey Belov
> */
> public class IgniteManagerTest {
> protected final Logger log =
> LoggerFactory.getLogger(IgniteManagerTest.class);
> private ThreadPoolExecutor ex = new ThreadPoolExecutor(8, 8,
> 0L, TimeUnit.MILLISECONDS,
> new LinkedBlockingQueue());
> @Test
> public void start() throws Exception {
> final IgniteConfiguration cfg = new IgniteConfiguration();
> final DataStorageConfiguration dataStorageConfiguration = new
> DataStorageConfiguration();
> final String igniteStorageDir = "g:\\work\\garbage\\igniteTest\\" +
> UUID.randomUUID().toString();
> System.out.println(igniteStorageDir);
> dataStorageConfiguration.setStoragePath(igniteStorageDir);
> final DataRegionConfiguration defaultDataRegionConfiguration =
> dataStorageConfiguration
> .getDefaultDataRegionConfiguration();
> defaultDataRegionConfiguration.setEvictionThreshold(0.9);
> defaultDataRegionConfiguration.setMetricsEnabled(true);
> defaultDataRegionConfiguration.setPersistenceEnabled(true);
> dataStorageConfiguration.setWalMode(WALMode.NONE);
> cfg.setDataStorageConfiguration(dataStorageConfiguration);
> final Ignite ignite = Ignition.start(cfg);
> ignite.cluster().active(true);
> while (true) {
> if (ex.getQueue().size() < 8) {
> System.out.println("added task " + ex.getQueue().size() + " "
> + ex.getActiveCount());
> ex.execute(() -> runQueues(ignite));
> }
> Thread.sleep(1000);
> }
> }
> private void qu() {
> }
> private void runQueues(Ignite ignite) {
> for (int j = 0; j < 10; j++) {
> final CollectionConfiguration setConfig = new
> CollectionConfiguration();
> setConfig.setCacheMode(CacheMode.LOCAL);
> setConfig.setBackups(0);
> final String name = "set-" + j + UUID.randomUUID().toString();
> setConfig.setGroupName(name);
> final IgniteSet set = ignite
> .set(name, setConfig);
> final int i1 = 1000;
> for (int i = 0; i < i1; i++) {
> final String elem1 = UUID.randomUUID().toString();
> set.add(elem1);
> }
> log.info(j + "write");
> set.clear();
> set.close();
> ignite.destroyCache(name);
> }
> log.info("Finish!");
> }
> }
> {code}
> See the attached screenshots from JProfiler.
> I think, that it should not be like this, because i clear the set and memory
> should be freed.
> If i launch this test with queue it works fine, memory becomes free after
> some time.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-10684) Memory leak in persistent IgniteSet
Alexey Belov created IGNITE-10684:
-
Summary: Memory leak in persistent IgniteSet
Key: IGNITE-10684
URL: https://issues.apache.org/jira/browse/IGNITE-10684
Project: Ignite
Issue Type: Bug
Components: data structures, persistence
Affects Versions: 2.7
Reporter: Alexey Belov
Fix For: None
Attachments: IgniteManagerTest.start - JProfiler 9.2.1 2.jpg,
IgniteManagerTest.start - JProfiler 9.2.1 3.jpg, IgniteManagerTest.start -
JProfiler 9.2.1.jpg
Hello. I have found a leak in IgniteSet with using persistence. Here is my Unit
Test:
{code:java}
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteSet;
import org.apache.ignite.Ignition;
import org.apache.ignite.cache.CacheMode;
import org.apache.ignite.configuration.*;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.UUID;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
/**
* @author Alexey Belov
*/
public class IgniteManagerTest {
protected final Logger log =
LoggerFactory.getLogger(IgniteManagerTest.class);
private ThreadPoolExecutor ex = new ThreadPoolExecutor(8, 8,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue());
@Test
public void start() throws Exception {
final IgniteConfiguration cfg = new IgniteConfiguration();
final DataStorageConfiguration dataStorageConfiguration = new
DataStorageConfiguration();
final String igniteStorageDir = "g:\\work\\garbage\\igniteTest\\" +
UUID.randomUUID().toString();
System.out.println(igniteStorageDir);
dataStorageConfiguration.setStoragePath(igniteStorageDir);
final DataRegionConfiguration defaultDataRegionConfiguration =
dataStorageConfiguration
.getDefaultDataRegionConfiguration();
defaultDataRegionConfiguration.setEvictionThreshold(0.9);
defaultDataRegionConfiguration.setMetricsEnabled(true);
defaultDataRegionConfiguration.setPersistenceEnabled(true);
dataStorageConfiguration.setWalMode(WALMode.NONE);
cfg.setDataStorageConfiguration(dataStorageConfiguration);
final Ignite ignite = Ignition.start(cfg);
ignite.cluster().active(true);
while (true) {
if (ex.getQueue().size() < 8) {
System.out.println("added task " + ex.getQueue().size() + " " +
ex.getActiveCount());
ex.execute(() -> runQueues(ignite));
}
Thread.sleep(1000);
}
}
private void qu() {
}
private void runQueues(Ignite ignite) {
for (int j = 0; j < 10; j++) {
final CollectionConfiguration setConfig = new
CollectionConfiguration();
setConfig.setCacheMode(CacheMode.LOCAL);
setConfig.setBackups(0);
final String name = "set-" + j + UUID.randomUUID().toString();
setConfig.setGroupName(name);
final IgniteSet set = ignite
.set(name, setConfig);
final int i1 = 1000;
for (int i = 0; i < i1; i++) {
final String elem1 = UUID.randomUUID().toString();
set.add(elem1);
}
log.info(j + "write");
set.clear();
set.close();
ignite.destroyCache(name);
}
log.info("Finish!");
}
}
{code}
See the attached screenshots from JProfiler.
I think, that it should not be like this, because i clear the set and memory
should be freed.
If i launch this test with queue it works fine, memory becomes free after some
time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
