[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[ https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16627053#comment-16627053 ] Peter Bacsko commented on OOZIE-3307: - Test failure are unrelated, +1 for this. > [core][oya] Limit heap usage of LauncherAM > -- > > Key: OOZIE-3307 > URL: https://issues.apache.org/jira/browse/OOZIE-3307 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0 >Reporter: Sabir Naikwadi >Assignee: Andras Piros >Priority: Critical > Fix For: 5.1.0 > > Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch, > OOZIE-3307.003.patch, OOZIE-3307.004.patch, OOZIE-3307.005.patch, > OOZIE-3307.006.patch > > > Application application_1531909575787_0039 failed 2 times due to AM Container > for appattempt_1531909575787_0039_02 exited with exitCode: -103 > Failing this attempt.Diagnostics: Container > [pid=11516,containerID=container_1531909575787_0039_02_01] is running > beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory > used; 10.0 GB of 10 GB virtual memory used. Killing container. > Dump of the process-tree for container_1531909575787_0039_02_01 : > | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE| > | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout > > 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr| > | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576 > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > For more detailed output, check the application tracking page: > [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039] > Then click on links to logs of each attempt. > . Failing the application.| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[
https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626128#comment-16626128
]
Hadoop QA commented on OOZIE-3307:
--
Testing JIRA OOZIE-3307
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 4 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:orange}0{color} There are [1] new bugs found in total that would be nice
to have fixed.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:orange}0{color} There are [1] new bugs found in [core] that would be
nice to have fixed.
. You can find the FindBugs diff here: core/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [webapp].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
There were errors during execution. Check console output for details.
.{color:orange}Tests failed at first run:{color}
TestJMSJobEventListener#testConnectionDrop
TestCallableQueueService#testPriorityExecutionOrder
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/855/
> [core][oya] Limit heap usage of LauncherAM
> --
>
> Key: OOZIE-3307
> URL: https://issues.apache.org/jira/browse/OOZIE-3307
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0
>Reporter: Sabir Naikwadi
>Assignee: Andras Piros
>Priority: Critical
> Fix For: 5.1.0
>
> Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch,
> OOZIE-3307.003.patch, OOZIE-3307.004.patch, OOZIE-3307.005.patch,
> OOZIE-3307.006.patch
>
>
> Application application_1531909575787_0039 failed 2 times due to AM Container
> for appattempt_1531909575787_0039_02 exited with exitCode: -103
> Failing this attempt.Diagnostics: Container
> [pid=11516,containerID=container_1531909575787_0039_02_01] is running
> beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory
> used; 10.0 GB of 10 GB virtual memory used. Killing container.
> Dump of the process-tree for container_1531909575787_0039_02_01 :
> | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[ https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625969#comment-16625969 ] Hadoop QA commented on OOZIE-3307: -- PreCommit-OOZIE-Build started > [core][oya] Limit heap usage of LauncherAM > -- > > Key: OOZIE-3307 > URL: https://issues.apache.org/jira/browse/OOZIE-3307 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0 >Reporter: Sabir Naikwadi >Assignee: Andras Piros >Priority: Critical > Fix For: 5.1.0 > > Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch, > OOZIE-3307.003.patch, OOZIE-3307.004.patch, OOZIE-3307.005.patch, > OOZIE-3307.006.patch > > > Application application_1531909575787_0039 failed 2 times due to AM Container > for appattempt_1531909575787_0039_02 exited with exitCode: -103 > Failing this attempt.Diagnostics: Container > [pid=11516,containerID=container_1531909575787_0039_02_01] is running > beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory > used; 10.0 GB of 10 GB virtual memory used. Killing container. > Dump of the process-tree for container_1531909575787_0039_02_01 : > | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE| > | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout > > 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr| > | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576 > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > For more detailed output, check the application tracking page: > [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039] > Then click on links to logs of each attempt. > . Failing the application.| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[
https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625963#comment-16625963
]
Andras Piros commented on OOZIE-3307:
-
Refactored code so that {{BytesAndUOMConverter}} is a separate class only being
used by {{LauncherMainTester}}, and user configured heap modifier options are
only checked for presence.
When present, maximum heap option isn't set. Former
{{LastParameterValueExtractor}} is deleted.
> [core][oya] Limit heap usage of LauncherAM
> --
>
> Key: OOZIE-3307
> URL: https://issues.apache.org/jira/browse/OOZIE-3307
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0
>Reporter: Sabir Naikwadi
>Assignee: Andras Piros
>Priority: Critical
> Fix For: 5.1.0
>
> Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch,
> OOZIE-3307.003.patch, OOZIE-3307.004.patch, OOZIE-3307.005.patch,
> OOZIE-3307.006.patch
>
>
> Application application_1531909575787_0039 failed 2 times due to AM Container
> for appattempt_1531909575787_0039_02 exited with exitCode: -103
> Failing this attempt.Diagnostics: Container
> [pid=11516,containerID=container_1531909575787_0039_02_01] is running
> beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory
> used; 10.0 GB of 10 GB virtual memory used. Killing container.
> Dump of the process-tree for container_1531909575787_0039_02_01 :
> | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE|
> | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c
> /usr/lib/jvm/java-openjdk/bin/java
> -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true
> -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01
> -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA
> -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev
> org.apache.oozie.action.hadoop.LauncherAM
> 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout
>
> 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr|
> | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576
> /usr/lib/jvm/java-openjdk/bin/java
> -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true
> -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01
> -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA
> -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev
> org.apache.oozie.action.hadoop.LauncherAM
> Container killed on request. Exit code is 143
> Container exited with a non-zero exit code 143
> For more detailed output, check the application tracking page:
> [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039]
> Then click on links to logs of each attempt.
> . Failing the application.|
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[
https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622331#comment-16622331
]
Hadoop QA commented on OOZIE-3307:
--
Testing JIRA OOZIE-3307
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 4 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:orange}0{color} There are [1] new bugs found in total that would be nice
to have fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:orange}0{color} There are [1] new bugs found in [core] that would be
nice to have fixed.
. You can find the FindBugs diff here: core/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
There were errors during execution. Check console output for details.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/842/
> [core][oya] Limit heap usage of LauncherAM
> --
>
> Key: OOZIE-3307
> URL: https://issues.apache.org/jira/browse/OOZIE-3307
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0
>Reporter: Sabir Naikwadi
>Assignee: Andras Piros
>Priority: Critical
> Fix For: 5.1.0
>
> Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch,
> OOZIE-3307.003.patch, OOZIE-3307.004.patch, OOZIE-3307.005.patch
>
>
> Application application_1531909575787_0039 failed 2 times due to AM Container
> for appattempt_1531909575787_0039_02 exited with exitCode: -103
> Failing this attempt.Diagnostics: Container
> [pid=11516,containerID=container_1531909575787_0039_02_01] is running
> beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory
> used; 10.0 GB of 10 GB virtual memory used. Killing container.
> Dump of the process-tree for container_1531909575787_0039_02_01 :
> | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE|
> | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c
> /usr/lib/jvm/java-openjdk/bin/java
> -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true
>
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[ https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622139#comment-16622139 ] Hadoop QA commented on OOZIE-3307: -- PreCommit-OOZIE-Build started > [core][oya] Limit heap usage of LauncherAM > -- > > Key: OOZIE-3307 > URL: https://issues.apache.org/jira/browse/OOZIE-3307 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0 >Reporter: Sabir Naikwadi >Assignee: Andras Piros >Priority: Critical > Fix For: 5.1.0 > > Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch, > OOZIE-3307.003.patch, OOZIE-3307.004.patch, OOZIE-3307.005.patch > > > Application application_1531909575787_0039 failed 2 times due to AM Container > for appattempt_1531909575787_0039_02 exited with exitCode: -103 > Failing this attempt.Diagnostics: Container > [pid=11516,containerID=container_1531909575787_0039_02_01] is running > beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory > used; 10.0 GB of 10 GB virtual memory used. Killing container. > Dump of the process-tree for container_1531909575787_0039_02_01 : > | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE| > | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout > > 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr| > | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576 > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > For more detailed output, check the application tracking page: > [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039] > Then click on links to logs of each attempt. > . Failing the application.| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[
https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622129#comment-16622129
]
Andras Piros commented on OOZIE-3307:
-
Removed method {{JavaActionExecutor#setOrRemoveConfiguredMaxHeap()}} and
changed behavior so that when there is a max heap configuration already set by
the user this will take precedence over the calculated one.
> [core][oya] Limit heap usage of LauncherAM
> --
>
> Key: OOZIE-3307
> URL: https://issues.apache.org/jira/browse/OOZIE-3307
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0
>Reporter: Sabir Naikwadi
>Assignee: Andras Piros
>Priority: Critical
> Fix For: 5.1.0
>
> Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch,
> OOZIE-3307.003.patch, OOZIE-3307.004.patch, OOZIE-3307.005.patch
>
>
> Application application_1531909575787_0039 failed 2 times due to AM Container
> for appattempt_1531909575787_0039_02 exited with exitCode: -103
> Failing this attempt.Diagnostics: Container
> [pid=11516,containerID=container_1531909575787_0039_02_01] is running
> beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory
> used; 10.0 GB of 10 GB virtual memory used. Killing container.
> Dump of the process-tree for container_1531909575787_0039_02_01 :
> | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE|
> | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c
> /usr/lib/jvm/java-openjdk/bin/java
> -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true
> -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01
> -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA
> -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev
> org.apache.oozie.action.hadoop.LauncherAM
> 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout
>
> 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr|
> | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576
> /usr/lib/jvm/java-openjdk/bin/java
> -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true
> -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01
> -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA
> -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev
> org.apache.oozie.action.hadoop.LauncherAM
> Container killed on request. Exit code is 143
> Container exited with a non-zero exit code 143
> For more detailed output, check the application tracking page:
> [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039]
> Then click on links to logs of each attempt.
> . Failing the application.|
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[
https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620537#comment-16620537
]
Hadoop QA commented on OOZIE-3307:
--
Testing JIRA OOZIE-3307
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 4 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
.{color:red}WARNING: the current HEAD has 1 RAT warning(s), they should be
addressed ASAP{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:orange}0{color} There are [1] new bugs found in total that would be nice
to have fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:orange}0{color} There are [1] new bugs found in [core] that would be
nice to have fixed.
. You can find the FindBugs diff here: core/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
There were errors during execution. Check console output for details.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/836/
> [core][oya] Limit heap usage of LauncherAM
> --
>
> Key: OOZIE-3307
> URL: https://issues.apache.org/jira/browse/OOZIE-3307
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0
>Reporter: Sabir Naikwadi
>Assignee: Andras Piros
>Priority: Critical
> Fix For: 5.1.0
>
> Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch,
> OOZIE-3307.003.patch, OOZIE-3307.004.patch
>
>
> Application application_1531909575787_0039 failed 2 times due to AM Container
> for appattempt_1531909575787_0039_02 exited with exitCode: -103
> Failing this attempt.Diagnostics: Container
> [pid=11516,containerID=container_1531909575787_0039_02_01] is running
> beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory
> used; 10.0 GB of 10 GB virtual memory used. Killing container.
> Dump of the process-tree for container_1531909575787_0039_02_01 :
> | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE|
> | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c
> /usr/lib/jvm/java-openjdk/bin/java
>
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[ https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620447#comment-16620447 ] Hadoop QA commented on OOZIE-3307: -- PreCommit-OOZIE-Build started > [core][oya] Limit heap usage of LauncherAM > -- > > Key: OOZIE-3307 > URL: https://issues.apache.org/jira/browse/OOZIE-3307 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0 >Reporter: Sabir Naikwadi >Assignee: Andras Piros >Priority: Critical > Fix For: 5.1.0 > > Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch, > OOZIE-3307.003.patch, OOZIE-3307.004.patch > > > Application application_1531909575787_0039 failed 2 times due to AM Container > for appattempt_1531909575787_0039_02 exited with exitCode: -103 > Failing this attempt.Diagnostics: Container > [pid=11516,containerID=container_1531909575787_0039_02_01] is running > beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory > used; 10.0 GB of 10 GB virtual memory used. Killing container. > Dump of the process-tree for container_1531909575787_0039_02_01 : > | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE| > | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout > > 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr| > | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576 > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > For more detailed output, check the application tracking page: > [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039] > Then click on links to logs of each attempt. > . Failing the application.| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[ https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620445#comment-16620445 ] Andras Piros commented on OOZIE-3307: - [~pbacsko] thanks for the review! Addressing your comments in patch 004. > [core][oya] Limit heap usage of LauncherAM > -- > > Key: OOZIE-3307 > URL: https://issues.apache.org/jira/browse/OOZIE-3307 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0 >Reporter: Sabir Naikwadi >Assignee: Andras Piros >Priority: Critical > Fix For: 5.1.0 > > Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch, > OOZIE-3307.003.patch, OOZIE-3307.004.patch > > > Application application_1531909575787_0039 failed 2 times due to AM Container > for appattempt_1531909575787_0039_02 exited with exitCode: -103 > Failing this attempt.Diagnostics: Container > [pid=11516,containerID=container_1531909575787_0039_02_01] is running > beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory > used; 10.0 GB of 10 GB virtual memory used. Killing container. > Dump of the process-tree for container_1531909575787_0039_02_01 : > | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE| > | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout > > 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr| > | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576 > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > For more detailed output, check the application tracking page: > [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039] > Then click on links to logs of each attempt. > . Failing the application.| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[
https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619294#comment-16619294
]
Hadoop QA commented on OOZIE-3307:
--
Testing JIRA OOZIE-3307
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 4 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
.{color:red}WARNING: the current HEAD has 1 RAT warning(s), they should be
addressed ASAP{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:orange}0{color} There are [1] new bugs found in total that would be nice
to have fixed.
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:orange}0{color} There are [1] new bugs found in [core] that would be
nice to have fixed.
. You can find the FindBugs diff here: core/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [tools].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
There were errors during execution. Check console output for details.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/832/
> [core][oya] Limit heap usage of LauncherAM
> --
>
> Key: OOZIE-3307
> URL: https://issues.apache.org/jira/browse/OOZIE-3307
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0
>Reporter: Sabir Naikwadi
>Assignee: Andras Piros
>Priority: Critical
> Fix For: 5.1.0
>
> Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch,
> OOZIE-3307.003.patch
>
>
> Application application_1531909575787_0039 failed 2 times due to AM Container
> for appattempt_1531909575787_0039_02 exited with exitCode: -103
> Failing this attempt.Diagnostics: Container
> [pid=11516,containerID=container_1531909575787_0039_02_01] is running
> beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory
> used; 10.0 GB of 10 GB virtual memory used. Killing container.
> Dump of the process-tree for container_1531909575787_0039_02_01 :
> | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE|
> | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c
> /usr/lib/jvm/java-openjdk/bin/java
> -Dlog4j.configuration=container-log4j.properties
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[ https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619088#comment-16619088 ] Hadoop QA commented on OOZIE-3307: -- PreCommit-OOZIE-Build started > [core][oya] Limit heap usage of LauncherAM > -- > > Key: OOZIE-3307 > URL: https://issues.apache.org/jira/browse/OOZIE-3307 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0 >Reporter: Sabir Naikwadi >Assignee: Andras Piros >Priority: Critical > Fix For: 5.1.0 > > Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch, > OOZIE-3307.003.patch > > > Application application_1531909575787_0039 failed 2 times due to AM Container > for appattempt_1531909575787_0039_02 exited with exitCode: -103 > Failing this attempt.Diagnostics: Container > [pid=11516,containerID=container_1531909575787_0039_02_01] is running > beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory > used; 10.0 GB of 10 GB virtual memory used. Killing container. > Dump of the process-tree for container_1531909575787_0039_02_01 : > | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE| > | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout > > 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr| > | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576 > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > For more detailed output, check the application tracking page: > [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039] > Then click on links to logs of each attempt. > . Failing the application.| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[
https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16619079#comment-16619079
]
Andras Piros commented on OOZIE-3307:
-
Thanks for the review [~gezapeti]! Uploaded a new patch that doesn't contain
unrelated {{pom.xml}} changes.
> [core][oya] Limit heap usage of LauncherAM
> --
>
> Key: OOZIE-3307
> URL: https://issues.apache.org/jira/browse/OOZIE-3307
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0
>Reporter: Sabir Naikwadi
>Assignee: Andras Piros
>Priority: Critical
> Fix For: 5.1.0
>
> Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch,
> OOZIE-3307.003.patch
>
>
> Application application_1531909575787_0039 failed 2 times due to AM Container
> for appattempt_1531909575787_0039_02 exited with exitCode: -103
> Failing this attempt.Diagnostics: Container
> [pid=11516,containerID=container_1531909575787_0039_02_01] is running
> beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory
> used; 10.0 GB of 10 GB virtual memory used. Killing container.
> Dump of the process-tree for container_1531909575787_0039_02_01 :
> | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE|
> | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c
> /usr/lib/jvm/java-openjdk/bin/java
> -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true
> -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01
> -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA
> -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev
> org.apache.oozie.action.hadoop.LauncherAM
> 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout
>
> 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr|
> | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576
> /usr/lib/jvm/java-openjdk/bin/java
> -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true
> -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01
> -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA
> -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev
> org.apache.oozie.action.hadoop.LauncherAM
> Container killed on request. Exit code is 143
> Container exited with a non-zero exit code 143
> For more detailed output, check the application tracking page:
> [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039]
> Then click on links to logs of each attempt.
> . Failing the application.|
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[ https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618982#comment-16618982 ] Peter Cseh commented on OOZIE-3307: --- Taking a quick look on the patch it looks like a fine improvement. +1 (pending jenkins) on it. There are some changes in the root pom.xml which are seemingly unrelated to this issue. Maybe they are leftovers from an other patch you're working on. Please remove them before commit, but don't wait for an other Jenkins build for it. > [core][oya] Limit heap usage of LauncherAM > -- > > Key: OOZIE-3307 > URL: https://issues.apache.org/jira/browse/OOZIE-3307 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0 >Reporter: Sabir Naikwadi >Assignee: Andras Piros >Priority: Critical > Fix For: 5.1.0 > > Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch > > > Application application_1531909575787_0039 failed 2 times due to AM Container > for appattempt_1531909575787_0039_02 exited with exitCode: -103 > Failing this attempt.Diagnostics: Container > [pid=11516,containerID=container_1531909575787_0039_02_01] is running > beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory > used; 10.0 GB of 10 GB virtual memory used. Killing container. > Dump of the process-tree for container_1531909575787_0039_02_01 : > | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE| > | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout > > 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr| > | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576 > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > For more detailed output, check the application tracking page: > [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039] > Then click on links to logs of each attempt. > . Failing the application.| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[ https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618983#comment-16618983 ] Peter Cseh commented on OOZIE-3307: --- Taking a quick look on the patch it looks like a fine improvement. +1 (pending jenkins) on it. There are some changes in the root pom.xml which are seemingly unrelated to this issue. Maybe they are leftovers from an other patch you're working on. Please remove them before commit, but don't wait for an other Jenkins build for it. > [core][oya] Limit heap usage of LauncherAM > -- > > Key: OOZIE-3307 > URL: https://issues.apache.org/jira/browse/OOZIE-3307 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0 >Reporter: Sabir Naikwadi >Assignee: Andras Piros >Priority: Critical > Fix For: 5.1.0 > > Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch > > > Application application_1531909575787_0039 failed 2 times due to AM Container > for appattempt_1531909575787_0039_02 exited with exitCode: -103 > Failing this attempt.Diagnostics: Container > [pid=11516,containerID=container_1531909575787_0039_02_01] is running > beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory > used; 10.0 GB of 10 GB virtual memory used. Killing container. > Dump of the process-tree for container_1531909575787_0039_02_01 : > | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE| > | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout > > 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr| > | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576 > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > For more detailed output, check the application tracking page: > [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039] > Then click on links to logs of each attempt. > . Failing the application.| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[ https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618816#comment-16618816 ] Hadoop QA commented on OOZIE-3307: -- PreCommit-OOZIE-Build started > [core][oya] Limit heap usage of LauncherAM > -- > > Key: OOZIE-3307 > URL: https://issues.apache.org/jira/browse/OOZIE-3307 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0 >Reporter: Sabir Naikwadi >Assignee: Andras Piros >Priority: Critical > Fix For: 5.1.0 > > Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch > > > Application application_1531909575787_0039 failed 2 times due to AM Container > for appattempt_1531909575787_0039_02 exited with exitCode: -103 > Failing this attempt.Diagnostics: Container > [pid=11516,containerID=container_1531909575787_0039_02_01] is running > beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory > used; 10.0 GB of 10 GB virtual memory used. Killing container. > Dump of the process-tree for container_1531909575787_0039_02_01 : > | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE| > | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout > > 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr| > | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576 > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > For more detailed output, check the application tracking page: > [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039] > Then click on links to logs of each attempt. > . Failing the application.| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[
https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618680#comment-16618680
]
Andras Piros commented on OOZIE-3307:
-
Addressed pre-commit issues:
* {{FINDBUGS_DIFF}} error because of complex regexp
* {{RAT}} errors on {{HEAD}} because of incorrect settings
> [core][oya] Limit heap usage of LauncherAM
> --
>
> Key: OOZIE-3307
> URL: https://issues.apache.org/jira/browse/OOZIE-3307
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.0.0
>Reporter: Sabir Naikwadi
>Assignee: Andras Piros
>Priority: Critical
> Fix For: 5.1.0
>
> Attachments: OOZIE-3307.001.patch, OOZIE-3307.002.patch
>
>
> Application application_1531909575787_0039 failed 2 times due to AM Container
> for appattempt_1531909575787_0039_02 exited with exitCode: -103
> Failing this attempt.Diagnostics: Container
> [pid=11516,containerID=container_1531909575787_0039_02_01] is running
> beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory
> used; 10.0 GB of 10 GB virtual memory used. Killing container.
> Dump of the process-tree for container_1531909575787_0039_02_01 :
> | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS)
> SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE|
> | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c
> /usr/lib/jvm/java-openjdk/bin/java
> -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true
> -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01
> -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA
> -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev
> org.apache.oozie.action.hadoop.LauncherAM
> 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout
>
> 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr|
> | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576
> /usr/lib/jvm/java-openjdk/bin/java
> -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true
> -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01
> -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA
> -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev
> org.apache.oozie.action.hadoop.LauncherAM
> Container killed on request. Exit code is 143
> Container exited with a non-zero exit code 143
> For more detailed output, check the application tracking page:
> [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039]
> Then click on links to logs of each attempt.
> . Failing the application.|
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[ https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16617676#comment-16617676 ] Andras Piros commented on OOZIE-3307: - [~pbacsko] there is a patch available, can you please review? Thanks! > [core][oya] Limit heap usage of LauncherAM > -- > > Key: OOZIE-3307 > URL: https://issues.apache.org/jira/browse/OOZIE-3307 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0 >Reporter: Sabir Naikwadi >Assignee: Andras Piros >Priority: Critical > Fix For: 5.1.0 > > Attachments: OOZIE-3307.001.patch > > > Application application_1531909575787_0039 failed 2 times due to AM Container > for appattempt_1531909575787_0039_02 exited with exitCode: -103 > Failing this attempt.Diagnostics: Container > [pid=11516,containerID=container_1531909575787_0039_02_01] is running > beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory > used; 10.0 GB of 10 GB virtual memory used. Killing container. > Dump of the process-tree for container_1531909575787_0039_02_01 : > | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE| > | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout > > 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr| > | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576 > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > For more detailed output, check the application tracking page: > [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039] > Then click on links to logs of each attempt. > . Failing the application.| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3307) [core][oya] Limit heap usage of LauncherAM
[ https://issues.apache.org/jira/browse/OOZIE-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16617672#comment-16617672 ] Hadoop QA commented on OOZIE-3307: -- PreCommit-OOZIE-Build started > [core][oya] Limit heap usage of LauncherAM > -- > > Key: OOZIE-3307 > URL: https://issues.apache.org/jira/browse/OOZIE-3307 > Project: Oozie > Issue Type: Bug > Components: core >Affects Versions: 5.0.0 >Reporter: Sabir Naikwadi >Assignee: Andras Piros >Priority: Critical > Fix For: 5.1.0 > > Attachments: OOZIE-3307.001.patch > > > Application application_1531909575787_0039 failed 2 times due to AM Container > for appattempt_1531909575787_0039_02 exited with exitCode: -103 > Failing this attempt.Diagnostics: Container > [pid=11516,containerID=container_1531909575787_0039_02_01] is running > beyond virtual memory limits. Current usage: 469.8 MB of 2 GB physical memory > used; 10.0 GB of 10 GB virtual memory used. Killing container. > Dump of the process-tree for container_1531909575787_0039_02_01 : > | - PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE| > | - 11516 11514 11516 11516 (bash) 1 3 115863552 682 /bin/bash -c > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > 1>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stdout > > 2>/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01/stderr| > | - 11755 11516 11516 11516 (java) 1142 71 10658242560 119576 > /usr/lib/jvm/java-openjdk/bin/java > -Dlog4j.configuration=container-log4j.properties -Dlog4j.debug=true > -Dyarn.app.container.log.dir=/var/log/hadoop-yarn/containers/application_1531909575787_0039/container_1531909575787_0039_02_01 > -Dyarn.app.container.log.filesize=1048576 -Dhadoop.root.logger=INFO,CLA > -Dhadoop.root.logfile=syslog -Dsubmitter.user=dev > org.apache.oozie.action.hadoop.LauncherAM > Container killed on request. Exit code is 143 > Container exited with a non-zero exit code 143 > For more detailed output, check the application tracking page: > [http://ip-10-20-201-36.us-gov-west-1.compute.internal:8088/cluster/app/application_1531909575787_0039] > Then click on links to logs of each attempt. > . Failing the application.| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
