Hi Pascal, 1,2. Please, read the article [1] for capacity planning. Ignite has an overhead on each entry for internal purposes. 3. Ignite is designed as distributed in-memory solution. Try to add more nodes, switch cache to PARTITIONED [2] mode and use data streamer [3] for populating the cache.
[1] https://apacheignite.readme.io/docs/capacity-planning [2] https://apacheignite.readme.io/docs/cache-modes#partitioned-mode [3] https://apacheignite.readme.io/docs/data-streamers#ignitedatastreamer On Sat, Jun 24, 2017 at 5:40 PM, Pascal <[email protected]> wrote: > Hi, > > I'm trying to test the singlethreaded-performance and memory consumption of > an Ignite cache using the following code: > > public class CachePerformance { > public static class PerformancePerson {public long id; public long > orgId; public String name; public int salary; > public PerformancePerson(long id, long orgId, String name, int > salary) { > this.id = id; this.orgId = orgId; this.name = name; > this.salary > = salary; > } > } > > public static void performanceTest() throws > java.lang.InterruptedException { > long personCount = 5000000; > Boolean onHeapCache = true; > > final String PERSON_CACHE = "myPersonCache"; > final long QUERY_COUNT = 1000000; > CacheConfiguration<Long, PerformancePerson> personCacheCfg = new > CacheConfiguration<>(PERSON_CACHE); > personCacheCfg.setCacheMode(CacheMode.LOCAL); > personCacheCfg.setOnheapCacheEnabled(onHeapCache); > personCacheCfg.setBackups(0); > IgniteConfiguration cfg = new IgniteConfiguration(); > cfg.setPublicThreadPoolSize(1); > Ignite ignite = Ignition.start(cfg); > IgniteCache<Long, PerformancePerson> personCache = > ignite.getOrCreateCache(personCacheCfg); > long usageEmpty = MemoryUtil.deepMemoryUsageOf(personCache); > Instant t1 = Instant.now(); > > for (long i = 0; i < personCount; i++) { > personCache.put(i, new PerformancePerson(i, i/10, "SampleName" > + > i, 10000 + (int) i * 10)); > } > Instant t2 = Instant.now(); > long usageFull = MemoryUtil.deepMemoryUsageOf(personCache); > > Instant t3 = Instant.now(); > for(long i=0; i< QUERY_COUNT; i++) { > long id = ThreadLocalRandom.current().nextLong(personCount); > PerformancePerson p = personCache.get(id); > } > Instant t4 = Instant.now(); > Duration writeDuration = Duration.between(t1, t2); > Duration readDuration = Duration.between(t3, t4); > System.out.println("OnHeap="+onHeapCache + " PersonCount=" + > personCount + > " avgItemSize=" + (usageFull - usageEmpty) / personCount + > " > " > +(1000L * personCount) / writeDuration.toMillis() + " > writes/second " > +(1000L * QUERY_COUNT) / readDuration.toMillis() +" > reads/second"); > } > } > > Result: > "C:\Program Files\Java\jdk1.8.0_45\bin\java" > [15:15:06] ver. 2.0.0#20170430-sha1:d4eef3c6 > [15:15:06] OS: Windows 7 6.1 amd64 > [15:15:10] Topology snapshot [ver=1, servers=1, clients=0, CPUs=12, > heap=7.1GB] > *OnHeap=true PersonCount=5000000 avgItemSize=332 183170 > writes/second > 647249 reads/second* > > A few questions I have: > 1) I use version 0.03 of Classmexer to estimate the size of the cache. Do > you know of a more reliable way to do this? I tried, JOL, but it doesn't > show the entire cache size. I explicitely want to include any indexes, > hashmaps and other (pre-)allocated space in the cache. > 2) Are these timings and sizes realistic for Ignite? The avgItemSize is > about twice as high as I would expect for such a simple POJO together with > a > hashmap. > 3) Can my code be improved to more reliably measure the real-world > performance of Ignite? > > Thank you very much for your help. > Pascal > > > > -- > View this message in context: http://apache-ignite-users. > 70518.x6.nabble.com/Measuring-cache-size-and-performance-tp14072.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. > -- Best regards, Andrey V. Mashenkov
