Hi Fvyaba,
I investigated your example. In your code you are going to create new cache
every time when you are going to create new table. Every new cache will have
some memory overhead. Next code can help you to get the average allocated
memory:
try (IgniteCache<?, ?> cache =
ignite.getOrCreateCache(defaultCacheCfg)) {
for(int i = 1; i < 100; i++) {
cache.query(new SqlFieldsQuery(String.format(
"CREATE TABLE TBL_%s (id BIGINT,uid VARCHAR,PRIMARY
KEY(id))", i)));
System.out.println("Count " + i + "
-------------------------------------------------------------");
for (DataRegionMetrics metrics :
ignite.dataRegionMetrics()) {
System.out.println(">>> Memory Region Name: " +
metrics.getName());
System.out.println(">>> Allocation Rate: " +
metrics.getAllocationRate());
System.out.println(">>> Allocated Size Full: " +
metrics.getTotalAllocatedSize());
System.out.println(">>> Allocated Size avg: " +
metrics.getTotalAllocatedSize() / i);
System.out.println(">>> Physical Memory Size: " +
metrics.getPhysicalMemorySize());
}
}
}
On my machine with default settings I got next:
>>> Memory Region Name: Default_Region
>>> Allocation Rate: 3419.9666
>>> Allocated Size Full: 840491008
>>> Allocated Size avg: 8489808
>>> Physical Memory Size: 840491008
So it's about 8mb per cache (so if you will have 3.2 GB then you can create
about 400 caches). I am not sure is it ok but you can do next to avoid
org.apache.ignite.IgniteCheckedException: Out of memory in data region:
1)Increase the max value of available off-heap memory:
<property name="dataStorageConfiguration">
<bean
class="org.apache.ignite.configuration.DataStorageConfiguration">
<property name="defaultDataRegionConfiguration">
<bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
<property name="name" value="Default_Region"/>
<property name="maxSize" value="#{1L * 1024 * 1024 *
1024}"/> //HERE
<property name="metricsEnabled" value="true"/>
</bean>
</property>
</bean>
</property>
2)Use persistence (or swaping space):
<property name="dataStorageConfiguration">
<bean
class="org.apache.ignite.configuration.DataStorageConfiguration">
<property name="defaultDataRegionConfiguration">
<bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
<property name="name" value="Default_Region"/>
<property name="maxSize" value="#{1L * 1024 * 1024 *
1024}"/>
<property name="metricsEnabled" value="true"/>
<property name="persistenceEnabled" value="true"/>
//THIS ONE
</bean>
</property>
</bean>
</property>
Read more about it you can here:
https://apacheignite.readme.io/docs/distributed-persistent-store
https://apacheignite.readme.io/v1.0/docs/off-heap-memory
Please try to test next code:
1) add this to your config:
<property name="dataStorageConfiguration">
<bean
class="org.apache.ignite.configuration.DataStorageConfiguration">
<property name="defaultDataRegionConfiguration">
<bean
class="org.apache.ignite.configuration.DataRegionConfiguration">
<property name="name" value="Default_Region"/>
<property name="maxSize" value="#{1L * 1024 * 1024 *
1024}"/>
<property name="metricsEnabled" value="true"/>
<property name="persistenceEnabled" value="true"/>
</bean>
</property>
</bean>
</property>
2)Run next:
public class example {
public static void main(String[] args) throws IgniteException {
try (Ignite ignite =
Ignition.start("examples/config/example-ignite.xml")) {
ignite.cluster().active(true);
CacheConfiguration<?, ?> defaultCacheCfg = new
CacheConfiguration<>("Default_cache").setSqlSchema("PUBLIC");
defaultCacheCfg.setDataRegionName("Default_Region");
try (IgniteCache<?, ?> cache =
ignite.getOrCreateCache(defaultCacheCfg)) {
for(int i = 1; i < 1000; i++) {
//remove old table cache just in case
cache.query(new SqlFieldsQuery(String.format(
"DROP TABLE TBL_%s", i)));
//create new table
cache.query(new SqlFieldsQuery(String.format(
"CREATE TABLE TBL_%s (id BIGINT,uid VARCHAR,PRIMARY
KEY(id))", i)));
System.out.println("Count " + i + "
-------------------------------------------------------------");
for (DataRegionMetrics metrics :
ignite.dataRegionMetrics()) {
System.out.println(">>> Memory Region Name: " +
metrics.getName());
System.out.println(">>> Allocation Rate: " +
metrics.getAllocationRate());
System.out.println(">>> Allocated Size Full: " +
metrics.getTotalAllocatedSize());
System.out.println(">>> Allocated Size avg: " +
metrics.getTotalAllocatedSize() / i);
System.out.println(">>> Physical Memory Size: " +
metrics.getPhysicalMemorySize());
}
}
}
ignite.cluster().active(false);
}
}
}
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
