Hi Neil, Thank you for code snippet. I was able to reproduce the problem and created a ticket - https://issues.apache.org/jira/browse/IGNITE-2833 Hope we will be able to fix it soon.
Vladimir. On Fri, Mar 11, 2016 at 10:13 AM, Neil Wightman <[email protected] > wrote: > Hi Vladimir, > > It took me a while to make a small test case but I now have one (see below) > > With the setExpiryPolicyFactory method enabled the heap after operation > 983Mb, with the setExpiryPolicyFactory commented out it is just 85Mb. > > Simple run this class file and once you see the console message attach > JVisualVM, then spam the "Perform GC" button. > > Also without the expiry time set the test runs with -Xm512m and with > expiry the test needs -Xmx2024m. > > Thanks > Neil > > -- IgniteExpiryIssue.java -- > package de.wightman.neil.apache.ignite.test; > > import javax.cache.configuration.FactoryBuilder; > import javax.cache.expiry.CreatedExpiryPolicy; > import javax.cache.expiry.Duration; > import org.apache.ignite.Ignite; > import org.apache.ignite.IgniteCache; > import org.apache.ignite.IgniteDataStreamer; > import org.apache.ignite.Ignition; > import org.apache.ignite.cache.CacheMemoryMode; > import org.apache.ignite.cache.CacheMode; > import org.apache.ignite.cache.eviction.lru.LruEvictionPolicy; > import org.apache.ignite.cache.query.annotations.QuerySqlField; > import org.apache.ignite.configuration.CacheConfiguration; > > public class IgniteExpiryIssue > { > > public static void main( final String[] args ) throws > InterruptedException > { > final long start = System.currentTimeMillis(); > // configures Tcp discovery for local host only > try (final Ignite ignite = Ignition.start( "LocalConfig.xml" )) > { > > final IgniteCache<CustomKey, CustomValue> cache = > ignite.getOrCreateCache( cacheConfig() ); > final IgniteDataStreamer<Object, Object> metricStreamer = > ignite.dataStreamer( "TestCacheName" ); > > final long counter = 1000; > final long counter2 = 49; > final long minutes = 60; > > final long fakeBaseTime = System.currentTimeMillis(); > > for( long k = 0; k < minutes; k++ ) > { > final long ts = fakeBaseTime + (minutes * 60 * 1000); > for( long i = 0; i < counter; i++ ) > { > for( long j = 0; j < counter2; j++ ) > { > final CustomKey key = new CustomKey( ts, i, j ); > > final CustomValue value = new CustomValue( 1.0, > 1.0, 1.0, 1.0, 10 ); > metricStreamer.addData( key, value ); > } > } > } > > // debug memory issue here > System.out.println("Duration = " + (System.currentTimeMillis() > - start)); > System.out.println("Cache filled attach JVisualVM to view > memory and generate a heap dump."); > Thread.sleep(Long.MAX_VALUE); > } > } > > > public static CacheConfiguration<CustomKey, CustomValue> cacheConfig() > { > final CacheConfiguration<CustomKey, CustomValue> > cacheConfiguration = new CacheConfiguration<>( "TestCacheName" ); > > cacheConfiguration.setBackups( 1 ); > cacheConfiguration.setCacheMode( CacheMode.PARTITIONED ); > cacheConfiguration.setIndexedTypes( CustomKey.class, > CustomValue.class ); > cacheConfiguration.setMemoryMode( CacheMemoryMode.OFFHEAP_TIERED ); > cacheConfiguration.setOffHeapMaxMemory( 4L * 1024L * 1024L * 1024L > ); > cacheConfiguration.setEvictionPolicy( new LruEvictionPolicy( > 6_000_000 ) ); > cacheConfiguration.setSwapEnabled( true ); > cacheConfiguration.setStatisticsEnabled( true ); > > // Causes heap usage to be high > cacheConfiguration.setExpiryPolicyFactory( > FactoryBuilder.factoryOf( new CreatedExpiryPolicy( Duration.ONE_HOUR ) ) ); > > return cacheConfiguration; > } > > public static class CustomKey > { > > @QuerySqlField(index = true, descending = false, name = "ts") > private long timeStamp; > > @QuerySqlField(index = true, name = "id1") > private long id1; > > @QuerySqlField(index = true, name = "id2") > private long id2; > > > public CustomKey( long timeStamp, > long id1, > long id2 ) > { > this.timeStamp = timeStamp; > this.id1 = id1; > this.id2 = id2; > } > } > > public static class CustomValue > { > > @QuerySqlField(name = "_d1") > private double d1; > @QuerySqlField() > private double d2; > @QuerySqlField() > private double d3; > @QuerySqlField() > private double d4; > @QuerySqlField(name = "_count") > private long count; > > > public CustomValue( double d1, > double d2, > double d3, > double d4, > long count ) > { > this.d1 = d1; > this.d2 = d2; > this.d3 = d3; > this.d4 = d4; > this.count = count; > } > } > } > ---- > > > On 10/03/16 13:04, Vladimir Ozerov wrote: > > Hi Neil, > > Could you please attach the code reproducing the problem? > > Vladimir. > > On Wed, Mar 9, 2016 at 1:16 PM, Neil Wightman <[email protected] > > wrote: > >> Hi All, >> >> I have been trying out ignite for the past few weeks but I am hitting a >> strange problem. I dont know if this problem is in my code or ignite. >> >> Currently I have an OFFHEAP_TIERED cache with ~3 million entries, max off >> heap size of 4Gb, but this is showing some very high heap memory usage. >> The cache stores data for 1 hour then expires it and constantly streams >> data into this cache via the dataStreamer.addData API. >> >> The problem seems to be with this specific expiry policy configuration : >> >> cacheConfiguration.setExpiryPolicyFactory( FactoryBuilder.factoryOf( new >> CreatedExpiryPolicy( Duration.ONE_HOUR ) ) ); >> >> - with this expiry policy the heap memory usage for the ignite process >> is 976Mb, without this expiry policy set its just 156Mb. >> >> The actual off heap memory usage is only 557Mb (assuming the cache >> metrics are correct). >> >> I have generated two heap dumps and the memory difference is all in the >> GridCacheTtlManager which is retaining a heap of ~889Mb according to >> Eclipses MAT. >> >> I have ran my tests with Ignite 1.5.0 Final and 1.6.0 SNAPSHOT from git >> (Monday 7th March) and both show the same issue. >> >> The problem with this large heap usage is that I am now getting very >> large GC pauses which I cant stop without disabling the expiry policy. >> >> My main question is am I doing something wrong? Is the Expiry Policy >> really supposed to use so much heap memory compared to the off heap cache >> size? >> >> Thanks in advance, >> Neil >> >> ---- config definition ---- >> >> final CacheConfiguration<MetricKey, AggregatePack> cacheConfiguration = >> new CacheConfiguration<>( "cacheName" ); >> cacheConfiguration.setBackups( 1 ); >> cacheConfiguration.setCacheMode( CacheMode.PARTITIONED ); >> cacheConfiguration.setIndexedTypes( CustomKey.class, CustomData.class ); >> cacheConfiguration.setMemoryMode( CacheMemoryMode.OFFHEAP_TIERED ); >> cacheConfiguration.setOffHeapMaxMemory( 4L * 1024L * 1024L * 1024L ); >> cacheConfiguration.setEvictionPolicy( new LruEvictionPolicy<>( 6_000_000 >> ) ); >> cacheConfiguration.setSwapEnabled( true ); >> cacheConfiguration.setStatisticsEnabled( true ); >> // This causes large memory usage >> cacheConfiguration.setExpiryPolicyFactory( FactoryBuilder.factoryOf( new >> CreatedExpiryPolicy( Duration.ONE_HOUR ) ) ); >> >> > >
