#1 I not sure if I got you point, as I known, Xmx is not turn into physical
memory as soon as the process running. it first loaded into virtual memory, if
you heap is need more, it will gradually increase in physical memory until to
the max heap.
#2 Physical memory contains not only heap, but also stack, direct memory,
shared lib, and perm space, and also there have VSS, RSS, PSS, USS concept, you
can google.
simple says:Vss = virtual set sizeRss = resident set sizePss = proportional set
size Uss = unique set size
Best Regards,Andy Hu(胡 珊)
Date: Fri, 29 May 2015 07:41:41 -0700
Subject: Re: Spark Executor Memory Usage
From: yuzhih...@gmail.com
To: valeramoisee...@gmail.com
CC: user@spark.apache.org
For #2, see
http://unix.stackexchange.com/questions/65835/htop-reporting-much-higher-memory-usage-than-free-or-top
Cheers
On Fri, May 29, 2015 at 6:56 AM, Valerii Moisieienko
valeramoisee...@gmail.com wrote:
Hello!
My name is Valerii. I have noticed strange memory behaivour of Spark's
executor on my cluster. Cluster works in standalone mode with 3 workers.
Application runs in cluster mode.
From topology configuration
spark.executor.memory 1536m
I checked heap usage via JVisualVM:
http://joxi.ru/Q2KqBMdSvYpDrj
and via htop:
http://joxi.ru/Vm63RWeCvG6L2Z
I have 2 questions regarding Spark's executors memory usage:
1. Why does Max Heap Size change during executor work?
2. Why does Memory usage via htop greater than executor's heap size?
Thank you!
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Executor-Memory-Usage-tp23083.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org