[
https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598887#comment-16598887
]
Vladimir Prus commented on OOZIE-3307:
--------------------------------------
Although this particular error message can be solved by tweaking yarn settings,
there is an issue with Oozie itself. It allocated 2GB Yarn container, however,
I don't see any Xmx option above. On many systems, default Xmx is above 2GB, so
with this default configuration, Ozzie AM can easily hit physical memory size
limit, and be likewise killed. Oozie 4 used to pass -Xmx1024 to AM by default.
Is this a issue that can be discussed in this Jira, or a new one would be more
appropriate?
> oozie workflow gets failed throwing error virtual memory limits
> ---------------------------------------------------------------
>
> Key: OOZIE-3307
> URL: https://issues.apache.org/jira/browse/OOZIE-3307
> Project: Oozie
> Issue Type: Bug
> Reporter: Sabir Naikwadi
> Priority: Critical
>
> Application application_1531909575787_0039 failed 2 times due to AM Container
> for appattempt_1531909575787_0039_000002 exited with exitCode: -103
> Failing this attempt.Diagnostics: Container
> [pid=11516,containerID=container_1531909575787_0039_02_000001] is running
> beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory
> used; 10.0 GB of 10 GB virtual memory used. Killing container.
> Dump of the process-tree for container_1531909575787_0039_02_000001 :
> | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE|
> | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c
> /usr/lib/jvm/java-openjdk/bin/java
> -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true
> -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_000001
> -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA
> -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev
> org.apache.oozie.action.hadoop.LauncherAM
> 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_000001/stdout
>
> 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_000001/stderr|
> | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576
> /usr/lib/jvm/java-openjdk/bin/java
> -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true
> -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_000001
> -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA
> -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev
> org.apache.oozie.action.hadoop.LauncherAM
> Container killed on request. Exit code is 143
> Container exited with a non-zero exit code 143
> For more detailed output, check the application tracking page:
> [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039]
> Then click on links to logs of each attempt.
> . Failing the application.|
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
