You might observe allocated rather than used memory space. How do you measure and how much of 8.5 gb is used by the off-heap storage vs. Java heap? Please check this page and let me know if you have other questions: https://www.gridgain.com/docs/latest/administrators-guide/monitoring-metrics/metrics#allocated-space-vs-actual-size-of-data
Denis On Thursday, November 21, 2019, Abdulrahman Semrie <[email protected]> wrote: > The objects are relatively large objects but not large enough to warrant > using 8.5GB of memory because loading the objects into memory from a file > without using Ignite doesn’t use that much memory. However, as you > mentioned if there is 200MB overhead for 1MB object then it might make > sense why it is using this much memory. > > Just as a comparison, I tried replacing Ignite with Hazelcast to store the > objects and it only consumed around 3GB of memory, 2x less than that of > Ignite. So this isn’t solely due to the object sizes. > > Here are my JVM options: > > -Xms512M > -Xmx8G > -XX:+UseParNewGC > -XX:+UseConcMarkSweepGC > -XX:+UseTLAB > -XX:NewSize=128m > -XX:MaxNewSize=128m > -XX:MaxTenuringThreshold=0 > -XX:SurvivorRatio=1024 > -XX:+UseCMSInitiatingOccupancyOnly > -XX:CMSInitiatingOccupancyFraction=40 > -XX:MaxGCPauseMillis=1000 > -XX:InitiatingHeapOccupancyPercent=50 > -XX:ParallelGCThreads=8 > -XX:ConcGCThreads=8 > -XX:+DisableExplicitGC > > I got most of the options from a similar thread discussing high memory > usage of Ignite and the OP of that thread suggested this JVM options helped > in reducing the usage by half but that didn’t work for my case :( > > Here is the thread -> http://apache-ignite-users. > 70518.x6.nabble.com/Very-high-memory-consumption-in-apache- > ignite-tp8822p8880.html > > Regards, > > Abdulrahman Semrie > On Nov 21, 2019, 2:13 PM +0300, Mikael <[email protected]>, wrote: > > Hi! > > Difficult to say without knowing anything about the data, you say "the > object", I assume you have lots of them ? > > How big is a serialized object, how many do you have ? > > Each entry in the cache have an overhead of around 200 bytes or so, so > 1M entries have an overhead of 200MB, pages are 4KB by default, if your > objects are big and use multiple pages you get some overhead there also. > > You say it use 8.5GB memory, how much is java heap and how much is other > memory ? > > Mikael > > Den 2019-11-21 kl. 11:01, skrev xabush: > > I have a serialized object that I load into memory to use it as a cache for > retrieving data. I am thinking of using Apache Ignite to scale my > application so that I can distribute the object over multiple nodes. > However, when trying to save the object in a single ignite cache, my > program > uses around 8.5GB of memory. Without Apache Ignite (i.e loading it directly > to memory and running computations on the object) my program uses only > 1.5GB > of memory. So I am wondering how is it using Ignite increases the memory > usage by 8x as that seems to be very high. And what can I do to reduce the > memory size? > > Here is my ignite configuration xml > https://pastebin.com/tejnqVK2 > > P.S I am using ignite with spring data. > > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > > -- - Denis
