: 2303480 *byte *Virtual Memory
Why does SparkSubmit process takes so much virtual memory in yarn-cluster
mode ? (which usually causes your Yarn container to be killed because of
outofmemory exception)
On Tue, Jul 14, 2015 at 9:39 AM, Elkhan Dadashov elkhan8...@gmail.com
wrote:
Hi all,
If you
Thanks, Marcelo.
That article confused me, thanks for correcting it helpful tips.
I looked into Virtual memory usage (jmap+jvisualvm) does not show that 11.5
g Virtual Memory usage - it is much less. I get 11.5 g Virtual memory usage
using top -p pid command for SparkSubmit process.
The
On Tue, Jul 14, 2015 at 3:42 PM, Elkhan Dadashov elkhan8...@gmail.com
wrote:
I looked into Virtual memory usage (jmap+jvisualvm) does not show that
11.5 g Virtual Memory usage - it is much less. I get 11.5 g Virtual memory
usage using top -p pid command for SparkSubmit process.
If you're
On Tue, Jul 14, 2015 at 9:53 AM, Elkhan Dadashov elkhan8...@gmail.com
wrote:
While the program is running, these are the stats of how much memory each
process takes:
SparkSubmit process : 11.266 *gigabyte* Virtual Memory
ApplicationMaster process: 2303480 *byte *Virtual Memory
That
