Hi Neil, Could you please attach the code reproducing the problem?
Vladimir. On Wed, Mar 9, 2016 at 1:16 PM, Neil Wightman <[email protected]> wrote: > Hi All, > > I have been trying out ignite for the past few weeks but I am hitting a > strange problem. I dont know if this problem is in my code or ignite. > > Currently I have an OFFHEAP_TIERED cache with ~3 million entries, max off > heap size of 4Gb, but this is showing some very high heap memory usage. > The cache stores data for 1 hour then expires it and constantly streams > data into this cache via the dataStreamer.addData API. > > The problem seems to be with this specific expiry policy configuration : > > cacheConfiguration.setExpiryPolicyFactory( FactoryBuilder.factoryOf( new > CreatedExpiryPolicy( Duration.ONE_HOUR ) ) ); > > - with this expiry policy the heap memory usage for the ignite process is > 976Mb, without this expiry policy set its just 156Mb. > > The actual off heap memory usage is only 557Mb (assuming the cache metrics > are correct). > > I have generated two heap dumps and the memory difference is all in the > GridCacheTtlManager which is retaining a heap of ~889Mb according to > Eclipses MAT. > > I have ran my tests with Ignite 1.5.0 Final and 1.6.0 SNAPSHOT from git > (Monday 7th March) and both show the same issue. > > The problem with this large heap usage is that I am now getting very large > GC pauses which I cant stop without disabling the expiry policy. > > My main question is am I doing something wrong? Is the Expiry Policy > really supposed to use so much heap memory compared to the off heap cache > size? > > Thanks in advance, > Neil > > ---- config definition ---- > > final CacheConfiguration<MetricKey, AggregatePack> cacheConfiguration = > new CacheConfiguration<>( "cacheName" ); > cacheConfiguration.setBackups( 1 ); > cacheConfiguration.setCacheMode( CacheMode.PARTITIONED ); > cacheConfiguration.setIndexedTypes( CustomKey.class, CustomData.class ); > cacheConfiguration.setMemoryMode( CacheMemoryMode.OFFHEAP_TIERED ); > cacheConfiguration.setOffHeapMaxMemory( 4L * 1024L * 1024L * 1024L ); > cacheConfiguration.setEvictionPolicy( new LruEvictionPolicy<>( 6_000_000 ) > ); > cacheConfiguration.setSwapEnabled( true ); > cacheConfiguration.setStatisticsEnabled( true ); > // This causes large memory usage > cacheConfiguration.setExpiryPolicyFactory( FactoryBuilder.factoryOf( new > CreatedExpiryPolicy( Duration.ONE_HOUR ) ) ); > >
