The objects are relatively large objects but not large enough to warrant using 8.5GB of memory because loading the objects into memory from a file without using Ignite doesn’t use that much memory. However, as you mentioned if there is 200MB overhead for 1MB object then it might make sense why it is using this much memory.
Just as a comparison, I tried replacing Ignite with Hazelcast to store the objects and it only consumed around 3GB of memory, 2x less than that of Ignite. So this isn’t solely due to the object sizes. Here are my JVM options: -Xms512M -Xmx8G -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+UseTLAB -XX:NewSize=128m -XX:MaxNewSize=128m -XX:MaxTenuringThreshold=0 -XX:SurvivorRatio=1024 -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=40 -XX:MaxGCPauseMillis=1000 -XX:InitiatingHeapOccupancyPercent=50 -XX:ParallelGCThreads=8 -XX:ConcGCThreads=8 -XX:+DisableExplicitGC I got most of the options from a similar thread discussing high memory usage of Ignite and the OP of that thread suggested this JVM options helped in reducing the usage by half but that didn’t work for my case :( Here is the thread -> http://apache-ignite-users.70518.x6.nabble.com/Very-high-memory-consumption-in-apache-ignite-tp8822p8880.html Regards, Abdulrahman Semrie On Nov 21, 2019, 2:13 PM +0300, Mikael <[email protected]>, wrote: > Hi! > > Difficult to say without knowing anything about the data, you say "the > object", I assume you have lots of them ? > > How big is a serialized object, how many do you have ? > > Each entry in the cache have an overhead of around 200 bytes or so, so > 1M entries have an overhead of 200MB, pages are 4KB by default, if your > objects are big and use multiple pages you get some overhead there also. > > You say it use 8.5GB memory, how much is java heap and how much is other > memory ? > > Mikael > > Den 2019-11-21 kl. 11:01, skrev xabush: > > I have a serialized object that I load into memory to use it as a cache for > > retrieving data. I am thinking of using Apache Ignite to scale my > > application so that I can distribute the object over multiple nodes. > > However, when trying to save the object in a single ignite cache, my program > > uses around 8.5GB of memory. Without Apache Ignite (i.e loading it directly > > to memory and running computations on the object) my program uses only 1.5GB > > of memory. So I am wondering how is it using Ignite increases the memory > > usage by 8x as that seems to be very high. And what can I do to reduce the > > memory size? > > > > Here is my ignite configuration xml > > https://pastebin.com/tejnqVK2 > > > > P.S I am using ignite with spring data. > > > > > > > > > > > > -- > > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > >
