Hi, guys,
I’m launching spark 2.2 job using oozie 4.1.0 (CDH 5.12.0). Oozie map task
(with shell script that started executed spark-submit command) fails after ~ 4h
of work with message in yarn:
Container [pid=49585,containerID=container_e112_1508189142310_0789_01_000002]
is running beyond physical memory limits. Current usage: 1.0 GB of 1 GB
physical memory used; 20.3 GB of 2.1 GB virtual memory used. Killing container.
I know that I can increase memory settings for oozie shell action, but i can’t
believe that need so much memory to run simple shell. Does anybody knows where
the root of the problem lays?
