Hi All,
I have been trying out ignite for the past few weeks but I am hitting a
strange problem. I dont know if this problem is in my code or ignite.
Currently I have an OFFHEAP_TIERED cache with ~3 million entries, max
off heap size of 4Gb, but this is showing some very high heap memory
usage. The cache stores data for 1 hour then expires it and constantly
streams data into this cache via the dataStreamer.addData API.
The problem seems to be with this specific expiry policy configuration :
cacheConfiguration.setExpiryPolicyFactory( FactoryBuilder.factoryOf( new
CreatedExpiryPolicy( Duration.ONE_HOUR ) ) );
- with this expiry policy the heap memory usage for the ignite process
is 976Mb, without this expiry policy set its just 156Mb.
The actual off heap memory usage is only 557Mb (assuming the cache
metrics are correct).
I have generated two heap dumps and the memory difference is all in the
GridCacheTtlManager which is retaining a heap of ~889Mb according to
Eclipses MAT.
I have ran my tests with Ignite 1.5.0 Final and 1.6.0 SNAPSHOT from git
(Monday 7th March) and both show the same issue.
The problem with this large heap usage is that I am now getting very
large GC pauses which I cant stop without disabling the expiry policy.
My main question is am I doing something wrong? Is the Expiry Policy
really supposed to use so much heap memory compared to the off heap
cache size?
Thanks in advance,
Neil
---- config definition ----
final CacheConfiguration<MetricKey, AggregatePack> cacheConfiguration =
new CacheConfiguration<>( "cacheName" );
cacheConfiguration.setBackups( 1 );
cacheConfiguration.setCacheMode( CacheMode.PARTITIONED );
cacheConfiguration.setIndexedTypes( CustomKey.class, CustomData.class );
cacheConfiguration.setMemoryMode( CacheMemoryMode.OFFHEAP_TIERED );
cacheConfiguration.setOffHeapMaxMemory( 4L * 1024L * 1024L * 1024L );
cacheConfiguration.setEvictionPolicy( new LruEvictionPolicy<>( 6_000_000
) );
cacheConfiguration.setSwapEnabled( true );
cacheConfiguration.setStatisticsEnabled( true );
// This causes large memory usage
cacheConfiguration.setExpiryPolicyFactory( FactoryBuilder.factoryOf( new
CreatedExpiryPolicy( Duration.ONE_HOUR ) ) );
