It's incorrect to use cache object to calculate cache data size. What you got now is a footprint of the Ignite infrastructure used to manage your data, but not a footprint of your data itself, since data are stored in off-heap and this tool only calculate on-heap size of objects referenced by cache object. As you can see in your sample 1 000 000 objects was created with 5 longs each, but you got only 2 872 685 longs in footprint.
To roughly estimate off-heap memory needed for cache data you can enable persistent, put large amount of entries to the cache and then measure size of cache folder (you should also wait until checkpoint complete to get correct results). Also page in off-heap memory have some extra header, so final off-heap memory needed will be a little bit larger then cache folder size. 2018-06-27 17:55 GMT+03:00 Prasad Bhalerao <prasadbhalerao1...@gmail.com>: > Hi, > > I have written a test program to find the cache size in bytes. I am using > this <https://jitpack.io/p/methodho/memory-measurer> tool > (memory-measurer) to find the object sizes and its foot prints. I tried to > use this tool to find the cache size in bytes. I am using on heap cache. > > Am I using the correct object (cache in this case) to check the cache size > in memory? Can some please clarify? > > I tried to use Jprofiler in instrumentation mode to check on-heap cache > size, but could not locate the exact class/objects. > > private void pushData(CacheName cacheName, List<? extends Data<? extends > DataKey>> datas) { > > final IgniteCache<DataKey, Data> cache = ignite.cache(cacheName.name()); > > LOGGER.info("cache size in bytes : {}", > IpV4RangeTest.bytesToHuman(MemoryMeasurer.measureBytes(cache))); > > TestDataObj data8 = null; > for(int ctr=0;ctr<1_000_000;ctr++){ > data8 = new TestDataObj(); > data8.setId(ctr); > data8.setSubscriptionId(SUBSCRIPTION_ID); > cache.put(data8.getKey(), data8); > } > > LOGGER.info("TestDataObj size in bytes : {}", > IpV4RangeTest.bytesToHuman(MemoryMeasurer.measureBytes(data8))); > > LOGGER.info("cache size in bytes : {}", > IpV4RangeTest.bytesToHuman(MemoryMeasurer.measureBytes(cache))); > > Footprint footprint = ObjectGraphMeasurer.measure(cache); > > LOGGER.info("{} Footprint={}", cacheName.name(), footprint.toString()); > > LOGGER.info("{} size={}", cacheName.name(), cache.size()); > try { > Thread.sleep(100000); > } catch (InterruptedException e) { > e.printStackTrace(); > } > } > > > > * Output:* > > *cache size in bytes* : 36.4 MB *[Empty Cache]* > > * TestDataObj size in bytes : 64 byte cache size in bytes* : 493.23 MB > *[After adding 1 million objects]* > *Footprint*=Footprint > [objects=10962325, > references=30951850, > primitives={byte=130277671, long=2872685, double=58, float=52015, > int=11651118, boolean=2156105, char=1905788, short=10457}] > > > *Inside TestDataObj class:* > > public class TestDataObj implements Data<DefaultDataAffinityKey> { > > @QuerySqlField > private long id; > > @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = > "ag_domain_nb_override_data", order = 1)}) > private long assetGroupDomainId; > > @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = > "ag_domain_nb_override_data", order = 2)}) > private long startIp; > > @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = > "ag_domain_nb_override_data", order = 3)}) > private long endIp; > > private long subscriptionId; > > @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = > "ag_domain_nb_override_data", order = 4)}) > private int partitionId; > > private boolean isUpdatedData; > > } > > > > Thanks, > PRasad > > On Wed, Jun 27, 2018 at 3:00 PM dkarachentsev <dkarachent...@gridgain.com> > wrote: > >> 1) This applicable to Ignite. As it grown from GridGain sometimes it may >> appear in docs, because missed fro removal. >> 2) Yes, and I would say overhead could be even bigger. But anyway I cannot >> say definitely how much, because Ignite doesn't store data sequentially, >> there a lot of nuances. >> 3) Ignite transaction caches entries on-heap and this works only for >> TRANSACTIONAL caches. >> >> Thanks! >> -Dmitry >> >> >> >> -- >> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >> >
