[jira] [Assigned] (YARN-7993) [UI2] yarn-service page need to consider ServiceState to show stop/delete buttons
[ https://issues.apache.org/jira/browse/YARN-7993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akhil PB reassigned YARN-7993: -- Assignee: Akhil PB (was: Gergely Novák) > [UI2] yarn-service page need to consider ServiceState to show stop/delete > buttons > - > > Key: YARN-7993 > URL: https://issues.apache.org/jira/browse/YARN-7993 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn-ui-v2 >Reporter: Sunil Govindan >Assignee: Akhil PB >Priority: Major > > yarn service page has stop/delete buttons. These buttons has to be > shown/hidden based on ServiceState of each app from ATS. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7833) [PERF/TEST] Extend SLS to support simulation of a Federated Environment
[ https://issues.apache.org/jira/browse/YARN-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16565919#comment-16565919 ] Tanuj Nayak edited comment on YARN-7833 at 8/3/18 4:34 AM: --- Can someone take a look at this [~curino] [~giovanni.fumarola] [~subru] [~botong] was (Author: tanujnay): Can someone take a look at this [~curino] [~giovanni.fumarola] [~subru] > [PERF/TEST] Extend SLS to support simulation of a Federated Environment > --- > > Key: YARN-7833 > URL: https://issues.apache.org/jira/browse/YARN-7833 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Carlo Curino >Assignee: Tanuj Nayak >Priority: Major > Attachments: YARN-7833.v1.patch, YARN-7833.v2.patch, > YARN-7833.v3.patch, YARN-7833.v4.patch, YARN-7833.v5.patch, > YARN-7833.v6.patch, YARN-7833.v7.patch > > > To develop algorithms for federation, it would be of great help to have a > version of SLS that supports multi RMs and GPG. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8505) AMLimit and userAMLimit check should be skipped for unmanaged AM
[ https://issues.apache.org/jira/browse/YARN-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567715#comment-16567715 ] Tao Yang commented on YARN-8505: Thanks [~suma.shivaprasad] for your comments. IIUC, unmanaged AM should not have a RMContainer instance, this patch can ignore these decAM/incAM handles for RMContainer because they are irrelated to unmanaged AM. Correct? > AMLimit and userAMLimit check should be skipped for unmanaged AM > > > Key: YARN-8505 > URL: https://issues.apache.org/jira/browse/YARN-8505 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler >Affects Versions: 3.2.0, 2.9.2 >Reporter: Tao Yang >Assignee: Tao Yang >Priority: Major > Attachments: YARN-8505.001.patch, YARN-8505.002.patch > > > AMLimit and userAMLimit check in LeafQueue#activateApplications should be > skipped for unmanaged AM whose resource is not taken from YARN cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8609) NM oom because of large container statuses
[ https://issues.apache.org/jira/browse/YARN-8609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567706#comment-16567706 ] Xianghao Lu commented on YARN-8609: --- Thanks for your comment. I have updated my patch according to your suggestion, but I find there is a same parameter(NM_CONTAINER_DIAGNOSTICS_MAXIMUM_SIZE) in YARN-3998. As far as I know, there are two kinds of diagnostics info, one is fixed string, such as "Container is killed before being launched.\n", the other is exception message which may be very large, so I think we should just truncate exception message rather than the entire string made by for loop. > NM oom because of large container statuses > -- > > Key: YARN-8609 > URL: https://issues.apache.org/jira/browse/YARN-8609 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Xianghao Lu >Priority: Major > Attachments: YARN-8609.001.patch, contain_status.jpg, oom.jpeg > > > Sometimes, NodeManger will send large container statuses to ResourceManager > when NodeManger start with recovering, as a result , NodeManger will be > failed to start because of oom. > In my case, the large container statuses size is 135M, which contain 11 > container statuses, and I find the diagnostics of 5 containers are very > large(27M), so, I truncate the container diagnostics as the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8609) NM oom because of large container statuses
[ https://issues.apache.org/jira/browse/YARN-8609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghao Lu updated YARN-8609: -- Attachment: YARN-8609.001.patch > NM oom because of large container statuses > -- > > Key: YARN-8609 > URL: https://issues.apache.org/jira/browse/YARN-8609 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Xianghao Lu >Priority: Major > Attachments: YARN-8609.001.patch, contain_status.jpg, oom.jpeg > > > Sometimes, NodeManger will send large container statuses to ResourceManager > when NodeManger start with recovering, as a result , NodeManger will be > failed to start because of oom. > In my case, the large container statuses size is 135M, which contain 11 > container statuses, and I find the diagnostics of 5 containers are very > large(27M), so, I truncate the container diagnostics as the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8609) NM oom because of large container statuses
[ https://issues.apache.org/jira/browse/YARN-8609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghao Lu updated YARN-8609: -- Attachment: (was: YARN-8609.001.patch) > NM oom because of large container statuses > -- > > Key: YARN-8609 > URL: https://issues.apache.org/jira/browse/YARN-8609 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Xianghao Lu >Priority: Major > Attachments: contain_status.jpg, oom.jpeg > > > Sometimes, NodeManger will send large container statuses to ResourceManager > when NodeManger start with recovering, as a result , NodeManger will be > failed to start because of oom. > In my case, the large container statuses size is 135M, which contain 11 > container statuses, and I find the diagnostics of 5 containers are very > large(27M), so, I truncate the container diagnostics as the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8609) NM oom because of large container statuses
[ https://issues.apache.org/jira/browse/YARN-8609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghao Lu updated YARN-8609: -- Attachment: YARN-8609.001.patch > NM oom because of large container statuses > -- > > Key: YARN-8609 > URL: https://issues.apache.org/jira/browse/YARN-8609 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Xianghao Lu >Priority: Major > Attachments: YARN-8609.001.patch, contain_status.jpg, oom.jpeg > > > Sometimes, NodeManger will send large container statuses to ResourceManager > when NodeManger start with recovering, as a result , NodeManger will be > failed to start because of oom. > In my case, the large container statuses size is 135M, which contain 11 > container statuses, and I find the diagnostics of 5 containers are very > large(27M), so, I truncate the container diagnostics as the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8609) NM oom because of large container statuses
[ https://issues.apache.org/jira/browse/YARN-8609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghao Lu updated YARN-8609: -- Attachment: (was: YARN-8609.001.patch) > NM oom because of large container statuses > -- > > Key: YARN-8609 > URL: https://issues.apache.org/jira/browse/YARN-8609 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Xianghao Lu >Priority: Major > Attachments: contain_status.jpg, oom.jpeg > > > Sometimes, NodeManger will send large container statuses to ResourceManager > when NodeManger start with recovering, as a result , NodeManger will be > failed to start because of oom. > In my case, the large container statuses size is 135M, which contain 11 > container statuses, and I find the diagnostics of 5 containers are very > large(27M), so, I truncate the container diagnostics as the patch. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7953) [GQ] Data structures for federation global queues calculations
[ https://issues.apache.org/jira/browse/YARN-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567688#comment-16567688 ] Abhishek Modi commented on YARN-7953: - Thanks [~botong]. Could you please review the latest patch. > [GQ] Data structures for federation global queues calculations > -- > > Key: YARN-7953 > URL: https://issues.apache.org/jira/browse/YARN-7953 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Carlo Curino >Assignee: Abhishek Modi >Priority: Major > Attachments: YARN-7953-YARN-7402.v1.patch, > YARN-7953-YARN-7402.v2.patch, YARN-7953-YARN-7402.v3.patch, > YARN-7953-YARN-7402.v4.patch, YARN-7953-YARN-7402.v5.patch, > YARN-7953-YARN-7402.v6.patch, YARN-7953-YARN-7402.v7.patch, YARN-7953.v1.patch > > > This Jira tracks data structures and helper classes used by the core > algorithms of YARN-7402 umbrella Jira (currently YARN-7403, and YARN-7834). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8607) Incorrect annotation in ApplicationAttemptStateData#getResourceSecondsMap
[
https://issues.apache.org/jira/browse/YARN-8607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yeliang Cang updated YARN-8607:
---
Description:
In ApplicationAttemptStateData.java
the annotation of getResourceSecondsMap is not correct:
{code:java}
/**
* Get the aggregated number of resources preempted that the application has
* allocated times the number of seconds the application has been running.
*
* @return map containing the resource name and aggregated preempted
* resource-seconds
*/
@Public
@Unstable
public abstract Map getResourceSecondsMap();
{code}
Should be
{code:java}
/**
* Get the aggregated number of resources that the application has
* allocated times the number of seconds the application has been running.
*
* @return map containing the resource name and aggregated
* resource-seconds
*/
@Public
@Unstable
public abstract Map getResourceSecondsMap();
{code}
was:
In ApplicationAttemptStateData.java
the annotation of getResourceSecondsMap is not correct:
{code}
/**
* Get the aggregated number of resources preempted that the application has
* allocated times the number of seconds the application has been running.
*
* @return map containing the resource name and aggregated preempted
* resource-seconds
*/
@Public
@Unstable
public abstract Map getResourceSecondsMap();
{code}
Should be
{code}
/**
* Get the aggregated number of resources that the application has
* allocated times the number of seconds the application has been running.
*
* @return map containing the resource name and aggregated preempted
* resource-seconds
*/
@Public
@Unstable
public abstract Map getResourceSecondsMap();
{code}
> Incorrect annotation in ApplicationAttemptStateData#getResourceSecondsMap
> -
>
> Key: YARN-8607
> URL: https://issues.apache.org/jira/browse/YARN-8607
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Reporter: Yeliang Cang
>Assignee: Yeliang Cang
>Priority: Trivial
> Attachments: YARN-8607.001.patch, YARN-8607.002.patch
>
>
> In ApplicationAttemptStateData.java
> the annotation of getResourceSecondsMap is not correct:
> {code:java}
> /**
> * Get the aggregated number of resources preempted that the application has
> * allocated times the number of seconds the application has been running.
> *
> * @return map containing the resource name and aggregated preempted
> * resource-seconds
> */
> @Public
> @Unstable
> public abstract Map getResourceSecondsMap();
> {code}
> Should be
> {code:java}
> /**
> * Get the aggregated number of resources that the application has
> * allocated times the number of seconds the application has been running.
> *
> * @return map containing the resource name and aggregated
> * resource-seconds
> */
> @Public
> @Unstable
> public abstract Map getResourceSecondsMap();
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8287) Update documentation and yarn-default related to the Docker runtime
[ https://issues.apache.org/jira/browse/YARN-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567658#comment-16567658 ] Craig Condit commented on YARN-8287: No tests as this is a documentation-only change. > Update documentation and yarn-default related to the Docker runtime > --- > > Key: YARN-8287 > URL: https://issues.apache.org/jira/browse/YARN-8287 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Shane Kumpf >Assignee: Craig Condit >Priority: Minor > Labels: Docker > Attachments: YARN-8287.001.patch > > > There are a few typos and omissions in the documentation and yarn-default wrt > running Docker containers on YARN. Below is what I noticed, but a more > thorough review is still needed: > * docker.allowed.volume-drivers is not documented > * None of the GPU or FPGA related items are in the Docker docs. > * "To run without any capabilites," - typo in yarn-default.xml > * remove from yarn-default.xml > * yarn.nodemanager.runtime.linux.docker.delayed-removal.allowed missing from > docs > * yarn.nodemanager.runtime.linux.docker.stop.grace-period missing from docs > * The user remapping features are missing from the docs, we should > explicitly call this out. > * The privileged container section could use a bit of rework to outline the > risks of the feature. > * Is it time to remove the security warnings? The community has made many > improvements since that warning was added. > * "path within the contatiner" typo -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8523) Interactive docker shell
[
https://issues.apache.org/jira/browse/YARN-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567619#comment-16567619
]
Eric Yang commented on YARN-8523:
-
[~divayjindal] One possible solution for option 3, embed xtermjs on node
manager UI, and implement a web socket servlet to forward data between docker
exec -it, and browser. The design looks like this:
{code}
xtermjs -> nodemanager web socket -> container-executor -> docker exec -it
{code}
> Interactive docker shell
>
>
> Key: YARN-8523
> URL: https://issues.apache.org/jira/browse/YARN-8523
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Eric Yang
>Priority: Major
> Labels: Docker
>
> Some application might require interactive unix commands executions to carry
> out operations. Container-executor can interface with docker exec to debug
> or analyze docker containers while the application is running. It would be
> nice to support an API to invoke docker exec to perform unix commands and
> report back the output to application master. Application master can
> distribute and aggregate execution of the commands to record in application
> master log file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8448) AM HTTPS Support
[
https://issues.apache.org/jira/browse/YARN-8448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567616#comment-16567616
]
genericqa commented on YARN-8448:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 11 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
37s{color} | {color:red} ozone-manager in trunk failed. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
23m 32s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
43s{color} | {color:red} hadoop-hdds/server-scm in trunk has 1 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
29s{color} | {color:red} ozone-manager in trunk failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m
5s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
19s{color} | {color:red} ozone-manager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 29m 49s{color} |
{color:red} root generated 1 new + 11 unchanged - 0 fixed = 12 total (was 11)
{color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 29m
49s{color} | {color:green} root generated 0 new + 1458 unchanged - 10 fixed =
1458 total (was 1468) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
38s{color} | {color:red} ozone-manager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
20s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 12m
19s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
29s{color} | {color:red} ozone-manager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
24s{color} | {color:green} hadoop-project in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
26s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
8s{color} |
[jira] [Commented] (YARN-8619) Automate docker network configuration through YARN API
[
https://issues.apache.org/jira/browse/YARN-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567603#comment-16567603
]
Eric Yang commented on YARN-8619:
-
[~shaneku...@gmail.com] "type" option can be merged as part of json similar to
"driver". In the cni example, it looks like other configuration file might be
required. In the spirit to keep everything look similar to yarn app command.
It might be more consistent to use JSON to reference to HDFS file to find
cni.cfg, or template string blob to embed the text file. This reduces the
parameters to pass to yarn cli. Thoughts?
> Automate docker network configuration through YARN API
> --
>
> Key: YARN-8619
> URL: https://issues.apache.org/jira/browse/YARN-8619
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn-native-services
>Reporter: Eric Yang
>Priority: Major
> Labels: Docker
>
> Docker supports bridge, host, overlay, macvlan networking. It might be
> useful to automate docker network setup through a set of YARN API to improve
> management of docker networks. Each type of network driver requires
> different type of parameters. For Hadoop use case, it seems more useful to
> focus on macvlan networking for ease of use and configuration. It would be
> great addition to support commands like:
> {code}
> yarn network create -d macvlan \
> --subnet=172.16.86.0/24 \
> --gateway=172.16.86.1 \
> -o parent=eth0 \
> my-macvlan-net
> {code}
> This changes docker configuration to hosts managed by YARN.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7974) Allow updating application tracking url after registration
[ https://issues.apache.org/jira/browse/YARN-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-7974: Attachment: YARN-7974-branch-2.001.patch > Allow updating application tracking url after registration > -- > > Key: YARN-7974 > URL: https://issues.apache.org/jira/browse/YARN-7974 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Fix For: 3.2.0, 3.0.4, 3.1.2 > > Attachments: YARN-7974-branch-2.001.patch, YARN-7974.001.patch, > YARN-7974.002.patch, YARN-7974.003.patch, YARN-7974.004.patch, > YARN-7974.005.patch, YARN-7974.006.patch > > > Normally an application's tracking url is set on AM registration. We have a > use case for updating the tracking url after registration (e.g. the UI is > hosted on one of the containers). > Approach is for AM to update tracking url on heartbeat to RM, and add related > API in AMRMClient. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7974) Allow updating application tracking url after registration
[ https://issues.apache.org/jira/browse/YARN-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-7974: Attachment: (was: YARN-7974-branch-2.001.patch) > Allow updating application tracking url after registration > -- > > Key: YARN-7974 > URL: https://issues.apache.org/jira/browse/YARN-7974 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Fix For: 3.2.0, 3.0.4, 3.1.2 > > Attachments: YARN-7974-branch-2.001.patch, YARN-7974.001.patch, > YARN-7974.002.patch, YARN-7974.003.patch, YARN-7974.004.patch, > YARN-7974.005.patch, YARN-7974.006.patch > > > Normally an application's tracking url is set on AM registration. We have a > use case for updating the tracking url after registration (e.g. the UI is > hosted on one of the containers). > Approach is for AM to update tracking url on heartbeat to RM, and add related > API in AMRMClient. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8619) Automate docker network configuration through YARN API
[
https://issues.apache.org/jira/browse/YARN-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567596#comment-16567596
]
Shane Kumpf commented on YARN-8619:
---
Thanks for the proposal, [~eyang]! I can see the value in making it easier to
admin container networking. However, I think we need to make this plugable
beyond Docker/macvlan, and I think we can do that with minimal changes to your
idea by adding a -type option or similar. One of the goals I'd like to see us
move towards is the adoption of prevailing standards, one of which is CNI,
which follows a different execution model than Docker's CNM/libnetwork and so
the macvlan options wouldn't apply. The -type option could lead to a CLI
similar to below:
Docker:
{code:java}
yarn network -create my-libnetwork-macvlan-net -type docker -conf
/tmp/network.json{code}
CNI:
{code:java}
yarn network -create my-cni-net -type cni -cni-config /tmp/cni.cfg -cni-plugin
/tmp/cni-plugin
{code}
As you mention, each host may need a different configuration. We may need a
-node option to target a specific nodemanager.
> Automate docker network configuration through YARN API
> --
>
> Key: YARN-8619
> URL: https://issues.apache.org/jira/browse/YARN-8619
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn-native-services
>Reporter: Eric Yang
>Priority: Major
> Labels: Docker
>
> Docker supports bridge, host, overlay, macvlan networking. It might be
> useful to automate docker network setup through a set of YARN API to improve
> management of docker networks. Each type of network driver requires
> different type of parameters. For Hadoop use case, it seems more useful to
> focus on macvlan networking for ease of use and configuration. It would be
> great addition to support commands like:
> {code}
> yarn network create -d macvlan \
> --subnet=172.16.86.0/24 \
> --gateway=172.16.86.1 \
> -o parent=eth0 \
> my-macvlan-net
> {code}
> This changes docker configuration to hosts managed by YARN.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8287) Update documentation and yarn-default related to the Docker runtime
[
https://issues.apache.org/jira/browse/YARN-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567566#comment-16567566
]
genericqa commented on YARN-8287:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
42m 47s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 18s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
14s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
19s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 69m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | YARN-8287 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12934180/YARN-8287.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient xml |
| uname | Linux 7e67b06b2bd1 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14
08:53:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 889df6f |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_171 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/21497/testReport/ |
| Max. process+thread count | 410 (vs. ulimit of 1) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/21497/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This
[jira] [Commented] (YARN-8619) Automate docker network configuration through YARN API
[
https://issues.apache.org/jira/browse/YARN-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567558#comment-16567558
]
Eric Yang commented on YARN-8619:
-
Each host may need to be configured differently, and using a file approach as
parameter might be more scalable to support other type of network driver as
well. Therefore, we might use:
{code}
yarn network -create my-macvlan-net /tmp/network.json
{code}
Content of json looks like:
{code}
{
"driver": "macvlan",
"hosts": [
{
"name":"host1.example.com",
"subnet": "172.16.86.0/24",
"gateway": "172.16.86.1",
"parent": "eth0"
}
]
}
{code}
For listing networks:
{code}
yarn network -list
{code}
For destroy networks:
{code}
yarn network -destroy my-macvlan-net
{code}
> Automate docker network configuration through YARN API
> --
>
> Key: YARN-8619
> URL: https://issues.apache.org/jira/browse/YARN-8619
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn-native-services
>Reporter: Eric Yang
>Priority: Major
> Labels: Docker
>
> Docker supports bridge, host, overlay, macvlan networking. It might be
> useful to automate docker network setup through a set of YARN API to improve
> management of docker networks. Each type of network driver requires
> different type of parameters. For Hadoop use case, it seems more useful to
> focus on macvlan networking for ease of use and configuration. It would be
> great addition to support commands like:
> {code}
> yarn network create -d macvlan \
> --subnet=172.16.86.0/24 \
> --gateway=172.16.86.1 \
> -o parent=eth0 \
> my-macvlan-net
> {code}
> This changes docker configuration to hosts managed by YARN.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7708) [GPG] Load based policy generator
[
https://issues.apache.org/jira/browse/YARN-7708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567532#comment-16567532
]
genericqa commented on YARN-7708:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} YARN-7402 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 28m
28s{color} | {color:green} YARN-7402 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
25s{color} | {color:green} YARN-7402 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} YARN-7402 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
25s{color} | {color:green} YARN-7402 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 26s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} YARN-7402 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
29s{color} | {color:green} YARN-7402 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 2s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}126m 7s{color}
| {color:red} hadoop-yarn in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 48s{color}
| {color:red} hadoop-yarn-server-globalpolicygenerator in the patch failed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}219m 57s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisher |
| | hadoop.yarn.server.globalpolicygenerator.TestGlobalPolicyGenerator |
| | hadoop.yarn.server.globalpolicygenerator.TestGlobalPolicyGenerator |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |
[jira] [Created] (YARN-8619) Automate docker network configuration through YARN API
Eric Yang created YARN-8619:
---
Summary: Automate docker network configuration through YARN API
Key: YARN-8619
URL: https://issues.apache.org/jira/browse/YARN-8619
Project: Hadoop YARN
Issue Type: Sub-task
Components: yarn-native-services
Reporter: Eric Yang
Docker supports bridge, host, overlay, macvlan networking. It might be useful
to automate docker network setup through a set of YARN API to improve
management of docker networks. Each type of network driver requires different
type of parameters. For Hadoop use case, it seems more useful to focus on
macvlan networking for ease of use and configuration. It would be great
addition to support commands like:
{code}
yarn network create -d macvlan \
--subnet=172.16.86.0/24 \
--gateway=172.16.86.1 \
-o parent=eth0 \
my-macvlan-net
{code}
This changes docker configuration to hosts managed by YARN.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8287) Update documentation and yarn-default related to the Docker runtime
[ https://issues.apache.org/jira/browse/YARN-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567514#comment-16567514 ] Craig Condit commented on YARN-8287: [~shaneku...@gmail.com], I'd like to work on this. > Update documentation and yarn-default related to the Docker runtime > --- > > Key: YARN-8287 > URL: https://issues.apache.org/jira/browse/YARN-8287 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Shane Kumpf >Priority: Minor > Labels: Docker > > There are a few typos and omissions in the documentation and yarn-default wrt > running Docker containers on YARN. Below is what I noticed, but a more > thorough review is still needed: > * docker.allowed.volume-drivers is not documented > * None of the GPU or FPGA related items are in the Docker docs. > * "To run without any capabilites," - typo in yarn-default.xml > * remove from yarn-default.xml > * yarn.nodemanager.runtime.linux.docker.delayed-removal.allowed missing from > docs > * yarn.nodemanager.runtime.linux.docker.stop.grace-period missing from docs > * The user remapping features are missing from the docs, we should > explicitly call this out. > * The privileged container section could use a bit of rework to outline the > risks of the feature. > * Is it time to remove the security warnings? The community has made many > improvements since that warning was added. > * "path within the contatiner" typo -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8287) Update documentation and yarn-default related to the Docker runtime
[ https://issues.apache.org/jira/browse/YARN-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Condit reassigned YARN-8287: -- Assignee: Craig Condit > Update documentation and yarn-default related to the Docker runtime > --- > > Key: YARN-8287 > URL: https://issues.apache.org/jira/browse/YARN-8287 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Shane Kumpf >Assignee: Craig Condit >Priority: Minor > Labels: Docker > > There are a few typos and omissions in the documentation and yarn-default wrt > running Docker containers on YARN. Below is what I noticed, but a more > thorough review is still needed: > * docker.allowed.volume-drivers is not documented > * None of the GPU or FPGA related items are in the Docker docs. > * "To run without any capabilites," - typo in yarn-default.xml > * remove from yarn-default.xml > * yarn.nodemanager.runtime.linux.docker.delayed-removal.allowed missing from > docs > * yarn.nodemanager.runtime.linux.docker.stop.grace-period missing from docs > * The user remapping features are missing from the docs, we should > explicitly call this out. > * The privileged container section could use a bit of rework to outline the > risks of the feature. > * Is it time to remove the security warnings? The community has made many > improvements since that warning was added. > * "path within the contatiner" typo -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8618) Yarn Service: When all the components of a service have restart policy NEVER then initiation of service upgrade should fail
[ https://issues.apache.org/jira/browse/YARN-8618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated YARN-8618: Description: * If all the components are NEVER, we can error out at the service level upgrade command itself and say upgrade not allowed. Otherwise for any other permutation/combination we should accept the service-level upgrade call and mark components which can be upgraded. > Yarn Service: When all the components of a service have restart policy NEVER > then initiation of service upgrade should fail > > > Key: YARN-8618 > URL: https://issues.apache.org/jira/browse/YARN-8618 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Chandni Singh >Assignee: Chandni Singh >Priority: Major > > * If all the components are NEVER, we can error out at the service level > upgrade command itself and say upgrade not allowed. Otherwise for any other > permutation/combination we should accept the service-level upgrade call and > mark components which can be upgraded. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8618) Yarn Service: When all the components of a service have restart policy NEVER then initiation of service upgrade should fail
Chandni Singh created YARN-8618: --- Summary: Yarn Service: When all the components of a service have restart policy NEVER then initiation of service upgrade should fail Key: YARN-8618 URL: https://issues.apache.org/jira/browse/YARN-8618 Project: Hadoop YARN Issue Type: Improvement Reporter: Chandni Singh Assignee: Chandni Singh -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8136) Add version attribute to site doc examples and quickstart
[
https://issues.apache.org/jira/browse/YARN-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567482#comment-16567482
]
genericqa commented on YARN-8136:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
35m 28s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 42s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | YARN-8136 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12933959/YARN-8136.001.patch |
| Optional Tests | asflicense mvnsite |
| uname | Linux 33ef4f1d2687 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14
08:53:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 889df6f |
| maven | version: Apache Maven 3.3.9 |
| Max. process+thread count | 417 (vs. ulimit of 1) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/21496/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Add version attribute to site doc examples and quickstart
> -
>
> Key: YARN-8136
> URL: https://issues.apache.org/jira/browse/YARN-8136
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: site
>Reporter: Gour Saha
>Priority: Major
> Attachments: YARN-8136.001.patch
>
>
> version attribute is missing in the following 2 site doc files -
> src/site/markdown/yarn-service/Examples.md
> src/site/markdown/yarn-service/QuickStart.md
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7863) Modify placement constraints to support node attributes
[
https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567462#comment-16567462
]
genericqa commented on YARN-7863:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} YARN-3409 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m
53s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
9s{color} | {color:green} YARN-3409 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
18s{color} | {color:green} YARN-3409 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} YARN-3409 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
54s{color} | {color:green} YARN-3409 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 52s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
11s{color} | {color:green} YARN-3409 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} YARN-3409 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 37s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
25s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
24s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
44s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 80m 43s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 40s{color}
| {color:red} hadoop-yarn-applications-distributedshell in the patch failed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}185m 46s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api |
| | Possible null pointer dereference of placementConstraints in
org.apache.hadoop.yarn.util.constraint.PlacementConstraintParser$NodeConstraintParser.parse()
Dereferenced at PlacementConstraintParser.java:placementConstraints in
[jira] [Commented] (YARN-8270) Adding JMX Metrics for Timeline Collector and Reader
[
https://issues.apache.org/jira/browse/YARN-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567442#comment-16567442
]
genericqa commented on YARN-8270:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 32m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 37s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 8s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
18s{color} | {color:green} hadoop-yarn-server-timelineservice in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 66m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | YARN-8270 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12934167/YARN-8270.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 0b4a64b99955 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 889df6f |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/21495/testReport/ |
| Max. process+thread count | 302 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/21495/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Adding JMX Metrics for Timeline Collector and
[jira] [Commented] (YARN-4946) RM should not consider an application as COMPLETED when log aggregation is not in a terminal state
[
https://issues.apache.org/jira/browse/YARN-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567434#comment-16567434
]
Robert Kanter commented on YARN-4946:
-
I'm not sure if any of the 3 versions of the ATS have the log aggregation
status info. But I agree that we shouldn't add this dependency if possible. I
also think it makes sense for the RM to remember Applications if they're still
doing something, including the log aggregation.
Thanks for the patch [~snemeth], a couple things:
# I'm not sure creating so many helper methods is necessary, especially the
ones that are one or two lines of code like {{recordLogAggregationStartTime}}.
# The current approach is changing when an App is considered finished
({{APP_COMPLETED}}) and delaying it until the log aggregation has finished.
That could take minutes after the App actually finishes, so this is going to
add a considerable delay on a bunch of other things - definitely something
users will notice. I think we should try to limit the scope of the changes so
that we leave the App lifecycle as-is, but only change the part where we decide
to evict an App from the RM.
#- More specifically, if you look at
{{RMAppManager#checkAppNumCompletedLimit}}, you can see that it's comparing a
counter for the number of completed apps vs the configured max. We can simply
adjust the logic here or the counter to only count an App once it's both
completed _and_ log aggregation has completed.
> RM should not consider an application as COMPLETED when log aggregation is
> not in a terminal state
> --
>
> Key: YARN-4946
> URL: https://issues.apache.org/jira/browse/YARN-4946
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: log-aggregation
>Affects Versions: 2.8.0
>Reporter: Robert Kanter
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-4946.001.patch, YARN-4946.002.patch
>
>
> MAPREDUCE-6415 added a tool that combines the aggregated log files for each
> Yarn App into a HAR file. When run, it seeds the list by looking at the
> aggregated logs directory, and then filters out ineligible apps. One of the
> criteria involves checking with the RM that an Application's log aggregation
> status is not still running and has not failed. When the RM "forgets" about
> an older completed Application (e.g. RM failover, enough time has passed,
> etc), the tool won't find the Application in the RM and will just assume that
> its log aggregation succeeded, even if it actually failed or is still running.
> We can solve this problem by doing the following:
> The RM should not consider an app to be fully completed (and thus removed
> from its history) until the aggregation status has reached a terminal state
> (e.g. SUCCEEDED, FAILED, TIME_OUT).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-8587) Delays are noticed to launch docker container
[
https://issues.apache.org/jira/browse/YARN-8587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Yang resolved YARN-8587.
-
Resolution: Not A Bug
> Delays are noticed to launch docker container
> -
>
> Key: YARN-8587
> URL: https://issues.apache.org/jira/browse/YARN-8587
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Yesha Vora
>Priority: Major
> Labels: Docker
>
> Launch dshell application. Wait for application to go in RUNNING state.
> {code:java}
> yarn jar /xx/hadoop-yarn-applications-distributedshell-*.jar -shell_command
> "sleep 300" -num_containers 1 -shell_env YARN_CONTAINER_RUNTIME_TYPE=docker
> -shell_env YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=httpd:0.1 -shell_env
> YARN_CONTAINER_RUNTIME_DOCKER_DELAYED_REMOVAL=true -jar
> /usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell-xx.jar
> {code}
> Find out container allocation. Run docker inspect command for docker
> containers launched by app.
> Sometimes, the container is allocated to NM but docker PID is not up.
> {code:java}
> Command ssh -q -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
> xxx "sudo su - -c \"docker ps -a | grep
> container_e02_1531189225093_0003_01_02\" root" failed after 0 retries
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8587) Delays are noticed to launch docker container
[
https://issues.apache.org/jira/browse/YARN-8587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567431#comment-16567431
]
Eric Yang commented on YARN-8587:
-
DistributedShell uses YARN v1 API, which doesn't support more fine-grained
status distinction between container-executor running vs docker running. If
docker run failed due to invalid parameters supplied by distributed shell, it
may take up to a minute to fail the container because the delay happens in
heart beat interval to report the status to AM and RM. The recommendation is
to update the test case to use yarn container -list [appId] to shorten the time
to check container running status from RM, but not completely eliminate
possible network delay in container status report.
> Delays are noticed to launch docker container
> -
>
> Key: YARN-8587
> URL: https://issues.apache.org/jira/browse/YARN-8587
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Yesha Vora
>Priority: Major
> Labels: Docker
>
> Launch dshell application. Wait for application to go in RUNNING state.
> {code:java}
> yarn jar /xx/hadoop-yarn-applications-distributedshell-*.jar -shell_command
> "sleep 300" -num_containers 1 -shell_env YARN_CONTAINER_RUNTIME_TYPE=docker
> -shell_env YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=httpd:0.1 -shell_env
> YARN_CONTAINER_RUNTIME_DOCKER_DELAYED_REMOVAL=true -jar
> /usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell-xx.jar
> {code}
> Find out container allocation. Run docker inspect command for docker
> containers launched by app.
> Sometimes, the container is allocated to NM but docker PID is not up.
> {code:java}
> Command ssh -q -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
> xxx "sudo su - -c \"docker ps -a | grep
> container_e02_1531189225093_0003_01_02\" root" failed after 0 retries
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8617) Aggregated Application Logs accumulates for long running jobs
Prabhu Joseph created YARN-8617: --- Summary: Aggregated Application Logs accumulates for long running jobs Key: YARN-8617 URL: https://issues.apache.org/jira/browse/YARN-8617 Project: Hadoop YARN Issue Type: New Feature Components: log-aggregation Affects Versions: 2.7.4 Reporter: Prabhu Joseph Currently AggregationDeletionService will delete older aggregated log files once when they are complete. This will cause logs to accumulate for Long Running Jobs like Llap, Spark Streaming. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8270) Adding JMX Metrics for Timeline Collector and Reader
[
https://issues.apache.org/jira/browse/YARN-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567415#comment-16567415
]
genericqa commented on YARN-8270:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
49s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 20s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 34s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
7s{color} | {color:green} hadoop-yarn-server-timelineservice in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 58m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | YARN-8270 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12934166/YARN-8270.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 0804edde9630 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14
08:53:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 889df6f |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/21494/testReport/ |
| Max. process+thread count | 440 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/21494/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Adding JMX Metrics for Timeline Collector and
[jira] [Commented] (YARN-8505) AMLimit and userAMLimit check should be skipped for unmanaged AM
[
https://issues.apache.org/jira/browse/YARN-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567413#comment-16567413
]
Suma Shivaprasad commented on YARN-8505:
[~Tao Yang] Thanks for the patch. Had a question/comment
Shouldnt we skip dec/inc resource usage for unManagedAMs in FicaSchedulerApp as
well?
{noformat}
public void nodePartitionUpdated(RMContainer rmContainer, String oldPartition,
String newPartition) {
Resource containerResource = rmContainer.getAllocatedResource();
this.attemptResourceUsage.decUsed(oldPartition, containerResource);
this.attemptResourceUsage.incUsed(newPartition, containerResource);
getCSLeafQueue().decUsedResource(oldPartition, containerResource, this);
getCSLeafQueue().incUsedResource(newPartition, containerResource, this);
// Update new partition name if container is AM and also update AM resource
if (rmContainer.isAMContainer()) {
setAppAMNodePartitionName(newPartition);
this.attemptResourceUsage.decAMUsed(oldPartition, containerResource);
this.attemptResourceUsage.incAMUsed(newPartition, containerResource);
getCSLeafQueue().decAMUsedResource(oldPartition, containerResource, this);
getCSLeafQueue().incAMUsedResource(newPartition, containerResource, this);
}
{noformat}
> AMLimit and userAMLimit check should be skipped for unmanaged AM
>
>
> Key: YARN-8505
> URL: https://issues.apache.org/jira/browse/YARN-8505
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
>Affects Versions: 3.2.0, 2.9.2
>Reporter: Tao Yang
>Assignee: Tao Yang
>Priority: Major
> Attachments: YARN-8505.001.patch, YARN-8505.002.patch
>
>
> AMLimit and userAMLimit check in LeafQueue#activateApplications should be
> skipped for unmanaged AM whose resource is not taken from YARN cluster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8403) Nodemanager logs failed to download file with INFO level
[
https://issues.apache.org/jira/browse/YARN-8403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Yang updated YARN-8403:
Issue Type: Bug (was: Sub-task)
Parent: (was: YARN-8472)
> Nodemanager logs failed to download file with INFO level
>
>
> Key: YARN-8403
> URL: https://issues.apache.org/jira/browse/YARN-8403
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Eric Yang
>Assignee: Eric Yang
>Priority: Major
> Fix For: 3.2.0, 3.1.2
>
> Attachments: YARN-8403.001.patch, YARN-8403.002.patch,
> YARN-8403.003.patch, YARN-8403.png
>
>
> Some of the container execution related stack traces are printing in INFO or
> WARN level.
> {code}
> 2018-06-06 03:10:40,077 INFO localizer.ResourceLocalizationService
> (ResourceLocalizationService.java:writeCredentials(1312)) - Writing
> credentials to the nmPrivate file
> /grid/0/hadoop/yarn/local/nmPrivate/container_e02_1528246317583_0048_01_01.tokens
> 2018-06-06 03:10:40,087 INFO localizer.ResourceLocalizationService
> (ResourceLocalizationService.java:run(975)) - Failed to download resource { {
> hdfs://mycluster.example.com:8020/user/hrt_qa/Streaming/InputDir,
> 1528254452720, FILE, null
> },pending,[(container_e02_1528246317583_0048_01_01)],6074418082915225,DOWNLOADING}
> org.apache.hadoop.yarn.exceptions.YarnException: Download and unpack failed
> at
> org.apache.hadoop.yarn.util.FSDownload.downloadAndUnpack(FSDownload.java:306)
> at
> org.apache.hadoop.yarn.util.FSDownload.verifyAndCopy(FSDownload.java:283)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:409)
> at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:66)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.FileNotFoundException:
> /grid/0/hadoop/yarn/local/filecache/28_tmp/InputDir/input1.txt (Permission
> denied)
> at java.io.FileOutputStream.open0(Native Method)
> at java.io.FileOutputStream.open(FileOutputStream.java:270)
> at java.io.FileOutputStream.(FileOutputStream.java:213)
> at
> org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.(RawLocalFileSystem.java:236)
> at
> org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.(RawLocalFileSystem.java:219)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.createOutputStreamWithMode(RawLocalFileSystem.java:318)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:307)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:338)
> at
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.(ChecksumFileSystem.java:401)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:464)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:443)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1169)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1149)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1038)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:408)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:399)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:381)
> at
> org.apache.hadoop.yarn.util.FSDownload.downloadAndUnpack(FSDownload.java:298)
> ... 9 more
> {code}
> {code}
> 2018-06-06 03:10:41,547 WARN privileged.PrivilegedOperationExecutor
> (PrivilegedOperationExecutor.java:executePrivilegedOperation(182)) -
> IOException executing command:
> java.io.InterruptedIOException: java.lang.InterruptedException
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:1012)
> at org.apache.hadoop.util.Shell.run(Shell.java:902)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.privileged.PrivilegedOperationExecutor.executePrivilegedOperation(PrivilegedOperationExecutor.java:152)
> at
> org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.startLocalizer(LinuxContainerExecutor.java:402)
> at
>
[jira] [Commented] (YARN-8509) Fix UserLimit calculation for preemption to balance scenario after queue satisfied
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567405#comment-16567405 ] Eric Payne commented on YARN-8509: -- I am very sorry for coming in late here. Please give me another day to review these changes. Maybe you have already covered this, but won't this change cause unnecessary preemption if pending > user limit > Fix UserLimit calculation for preemption to balance scenario after queue > satisfied > > > Key: YARN-8509 > URL: https://issues.apache.org/jira/browse/YARN-8509 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Zian Chen >Assignee: Zian Chen >Priority: Major > Attachments: YARN-8509.001.patch, YARN-8509.002.patch, > YARN-8509.003.patch > > > In LeafQueue#getTotalPendingResourcesConsideringUserLimit, we calculate total > pending resource based on user-limit percent and user-limit factor which will > cap pending resource for each user to the minimum of user-limit pending and > actual pending. This will prevent queue from taking more pending resource to > achieve queue balance after all queue satisfied with its ideal allocation. > > We need to change the logic to let queue pending can go beyond userlimit. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart
[
https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haibo Chen updated YARN-6966:
-
Fix Version/s: 2.10.0
> NodeManager metrics may return wrong negative values when NM restart
>
>
> Key: YARN-6966
> URL: https://issues.apache.org/jira/browse/YARN-6966
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Yang Wang
>Assignee: Szilard Nemeth
>Priority: Major
> Fix For: 2.10.0, 3.2.0, 3.0.4, 3.1.2
>
> Attachments: YARN-6966-branch-2.001.patch,
> YARN-6966-branch-2.002.patch, YARN-6966-branch-2.002.patch,
> YARN-6966-branch-2.002.patch, YARN-6966-branch-3.0.0.001.patch,
> YARN-6966-branch-3.0.001.patch, YARN-6966.001.patch, YARN-6966.002.patch,
> YARN-6966.003.patch, YARN-6966.004.patch, YARN-6966.005.patch,
> YARN-6966.005.patch, YARN-6966.006.patch
>
>
> Just as YARN-6212. However, I think it is not a duplicate of YARN-3933.
> The primary cause of negative values is that metrics do not recover properly
> when NM restart.
> AllocatedContainers,ContainersLaunched,AllocatedGB,AvailableGB,AllocatedVCores,AvailableVCores
> in metrics also need to recover when NM restart.
> This should be done in ContainerManagerImpl#recoverContainer.
> The scenario could be reproduction by the following steps:
> # Make sure
> YarnConfiguration.NM_RECOVERY_ENABLED=true,YarnConfiguration.NM_RECOVERY_SUPERVISED=true
> in NM
> # Submit an application and keep running
> # Restart NM
> # Stop the application
> # Now you get the negative values
> {code}
> /jmx?qry=Hadoop:service=NodeManager,name=NodeManagerMetrics
> {code}
> {code}
> {
> name: "Hadoop:service=NodeManager,name=NodeManagerMetrics",
> modelerType: "NodeManagerMetrics",
> tag.Context: "yarn",
> tag.Hostname: "hadoop.com",
> ContainersLaunched: 0,
> ContainersCompleted: 0,
> ContainersFailed: 2,
> ContainersKilled: 0,
> ContainersIniting: 0,
> ContainersRunning: 0,
> AllocatedGB: 0,
> AllocatedContainers: -2,
> AvailableGB: 160,
> AllocatedVCores: -11,
> AvailableVCores: 3611,
> ContainerLaunchDurationNumOps: 2,
> ContainerLaunchDurationAvgTime: 6,
> BadLocalDirs: 0,
> BadLogDirs: 0,
> GoodLocalDirsDiskUtilizationPerc: 2,
> GoodLogDirsDiskUtilizationPerc: 2
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart
[
https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567394#comment-16567394
]
Haibo Chen commented on YARN-6966:
--
The build has finished at
[https://builds.apache.org/job/PreCommit-YARN-Build/21489/console.] But somehow
the report was not posted here.
Checking the branch-2 in shortly.
> NodeManager metrics may return wrong negative values when NM restart
>
>
> Key: YARN-6966
> URL: https://issues.apache.org/jira/browse/YARN-6966
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Yang Wang
>Assignee: Szilard Nemeth
>Priority: Major
> Fix For: 3.2.0, 3.0.4, 3.1.2
>
> Attachments: YARN-6966-branch-2.001.patch,
> YARN-6966-branch-2.002.patch, YARN-6966-branch-2.002.patch,
> YARN-6966-branch-2.002.patch, YARN-6966-branch-3.0.0.001.patch,
> YARN-6966-branch-3.0.001.patch, YARN-6966.001.patch, YARN-6966.002.patch,
> YARN-6966.003.patch, YARN-6966.004.patch, YARN-6966.005.patch,
> YARN-6966.005.patch, YARN-6966.006.patch
>
>
> Just as YARN-6212. However, I think it is not a duplicate of YARN-3933.
> The primary cause of negative values is that metrics do not recover properly
> when NM restart.
> AllocatedContainers,ContainersLaunched,AllocatedGB,AvailableGB,AllocatedVCores,AvailableVCores
> in metrics also need to recover when NM restart.
> This should be done in ContainerManagerImpl#recoverContainer.
> The scenario could be reproduction by the following steps:
> # Make sure
> YarnConfiguration.NM_RECOVERY_ENABLED=true,YarnConfiguration.NM_RECOVERY_SUPERVISED=true
> in NM
> # Submit an application and keep running
> # Restart NM
> # Stop the application
> # Now you get the negative values
> {code}
> /jmx?qry=Hadoop:service=NodeManager,name=NodeManagerMetrics
> {code}
> {code}
> {
> name: "Hadoop:service=NodeManager,name=NodeManagerMetrics",
> modelerType: "NodeManagerMetrics",
> tag.Context: "yarn",
> tag.Hostname: "hadoop.com",
> ContainersLaunched: 0,
> ContainersCompleted: 0,
> ContainersFailed: 2,
> ContainersKilled: 0,
> ContainersIniting: 0,
> ContainersRunning: 0,
> AllocatedGB: 0,
> AllocatedContainers: -2,
> AvailableGB: 160,
> AllocatedVCores: -11,
> AvailableVCores: 3611,
> ContainerLaunchDurationNumOps: 2,
> ContainerLaunchDurationAvgTime: 6,
> BadLocalDirs: 0,
> BadLogDirs: 0,
> GoodLocalDirsDiskUtilizationPerc: 2,
> GoodLogDirsDiskUtilizationPerc: 2
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8608) [UI2] No information available per application appAttempt about 'Total Outstanding Resource Requests'
[ https://issues.apache.org/jira/browse/YARN-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8608: - Reporter: Sumana Sathish (was: Akhil PB) > [UI2] No information available per application appAttempt about 'Total > Outstanding Resource Requests' > - > > Key: YARN-8608 > URL: https://issues.apache.org/jira/browse/YARN-8608 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Reporter: Sumana Sathish >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8608.001.patch > > > There is no information available about the 'Total Outstanding Resource > Requests' in new RM UI. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8615) [UI2] Resource Usage tab shows only memory related info. No info available for vcores/gpu.
[ https://issues.apache.org/jira/browse/YARN-8615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8615: - Reporter: Sumana Sathish (was: Akhil PB) > [UI2] Resource Usage tab shows only memory related info. No info available > for vcores/gpu. > -- > > Key: YARN-8615 > URL: https://issues.apache.org/jira/browse/YARN-8615 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Reporter: Sumana Sathish >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8615.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8270) Adding JMX Metrics for Timeline Collector and Reader
[ https://issues.apache.org/jira/browse/YARN-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sushil Ks updated YARN-8270: Attachment: (was: YARN-8270.002.patch) > Adding JMX Metrics for Timeline Collector and Reader > > > Key: YARN-8270 > URL: https://issues.apache.org/jira/browse/YARN-8270 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2, timelineserver >Reporter: Sushil Ks >Assignee: Sushil Ks >Priority: Major > Attachments: YARN-8270.001.patch > > > This Jira is for emitting JMX Metrics for ATS v2 Timeline Collector and > Timeline Reader, basically for Timeline Collector it tries to capture > success, failure and latencies for *putEntities* and *putEntitiesAsync* from > *TimelineCollectorWebService* and all the API's success, failure and > latencies for fetching TimelineEntities from *TimelineReaderWebServices*. > This would actually help in monitoring and measuring performance for ATSv2 at > scale. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8549) Adding a NoOp timeline writer and reader plugin classes for ATSv2
[
https://issues.apache.org/jira/browse/YARN-8549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567342#comment-16567342
]
genericqa commented on YARN-8549:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
19s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
33s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 15s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice:
The patch generated 115 new + 0 unchanged - 0 fixed = 115 total (was 0)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 62m 37s{color}
| {color:red} hadoop-yarn-server-timelineservice in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
24s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 83m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:a716388 |
| JIRA Issue | YARN-8549 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12934103/YARN-8549-branch-2.03.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux ef75f0670f5e 3.13.0-144-generic #193-Ubuntu SMP Thu Mar 15
17:03:53 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | branch-2 / e2b82b8 |
| maven | version: Apache Maven 3.3.9
(bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) |
| Default Java | 1.7.0_181 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/21490/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice.txt
|
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/21490/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/21490/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-YARN-Build/21490/artifact/out/patch-asflicense-problems.txt
|
| Max. process+thread count | 91 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice
|
| Console output |
[jira] [Commented] (YARN-7953) [GQ] Data structures for federation global queues calculations
[
https://issues.apache.org/jira/browse/YARN-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567327#comment-16567327
]
genericqa commented on YARN-7953:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
|| || || || {color:brown} YARN-7402 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
58s{color} | {color:green} YARN-7402 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} YARN-7402 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} YARN-7402 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} YARN-7402 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 15s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} YARN-7402 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} YARN-7402 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 29s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
36s{color} | {color:green} hadoop-yarn-server-globalpolicygenerator in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 58m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | YARN-7953 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12933895/YARN-7953-YARN-7402.v7.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient xml findbugs checkstyle |
| uname | Linux e142d3c01cff 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | YARN-7402 / b8e7180 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/21491/testReport/ |
| Max. process+thread count | 311 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator
|
| Console output |
[jira] [Commented] (YARN-7953) [GQ] Data structures for federation global queues calculations
[ https://issues.apache.org/jira/browse/YARN-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567130#comment-16567130 ] Botong Huang commented on YARN-7953: Rebase done and re-kicked yetus for this thread. > [GQ] Data structures for federation global queues calculations > -- > > Key: YARN-7953 > URL: https://issues.apache.org/jira/browse/YARN-7953 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Carlo Curino >Assignee: Abhishek Modi >Priority: Major > Attachments: YARN-7953-YARN-7402.v1.patch, > YARN-7953-YARN-7402.v2.patch, YARN-7953-YARN-7402.v3.patch, > YARN-7953-YARN-7402.v4.patch, YARN-7953-YARN-7402.v5.patch, > YARN-7953-YARN-7402.v6.patch, YARN-7953-YARN-7402.v7.patch, YARN-7953.v1.patch > > > This Jira tracks data structures and helper classes used by the core > algorithms of YARN-7402 umbrella Jira (currently YARN-7403, and YARN-7834). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567127#comment-16567127 ] Sunil Govindan commented on YARN-7863: -- Attached rebased patch. [~cheersyang], these constraints are treated as independently in scheduler. For each request on the lookup node, we can check the placement request which is demanded. And if multiple constraints are made on one requests, it will be checked sequentially. Current design is inline with placement spec design what is done. But i think AND mapped to : might have created a bit of confusion. > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7863) Modify placement constraints to support node attributes
[ https://issues.apache.org/jira/browse/YARN-7863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7863: - Attachment: YARN-7863-YARN-3409.004.patch > Modify placement constraints to support node attributes > --- > > Key: YARN-7863 > URL: https://issues.apache.org/jira/browse/YARN-7863 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sunil Govindan >Assignee: Sunil Govindan >Priority: Major > Attachments: YARN-7863-YARN-3409.002.patch, > YARN-7863-YARN-3409.003.patch, YARN-7863-YARN-3409.004.patch, > YARN-7863.v0.patch > > > This Jira will track to *Modify existing placement constraints to support > node attributes.* -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7159) Normalize unit of resource objects in RM to avoid unit conversion in critical path
[ https://issues.apache.org/jira/browse/YARN-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567120#comment-16567120 ] Hudson commented on YARN-7159: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14699 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14699/]) YARN-7159. Normalize unit of resource objects in RM to avoid unit (sunilg: rev 12a095a496dd59066d73a7a6c24129b5b6a9d650) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/resource/TestResourceUtils.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/resource/DominantResourceCalculator.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/TestResourcePBImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ResourceInformation.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourcePBImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/test/java/org/apache/hadoop/yarn/conf/TestResourceInformation.java * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/resources/resource-types/node-resources-3.xml * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/util/resource/ResourceUtils.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairSchedulerConfiguration.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/resource/Resources.java > Normalize unit of resource objects in RM to avoid unit conversion in critical > path > -- > > Key: YARN-7159 > URL: https://issues.apache.org/jira/browse/YARN-7159 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Wangda Tan >Assignee: Manikandan R >Priority: Critical > Fix For: 3.2.0 > > Attachments: YARN-7159.001.patch, YARN-7159.002.patch, > YARN-7159.003.patch, YARN-7159.004.patch, YARN-7159.005.patch, > YARN-7159.006.patch, YARN-7159.007.patch, YARN-7159.008.patch, > YARN-7159.009.patch, YARN-7159.010.patch, YARN-7159.011.patch, > YARN-7159.012.patch, YARN-7159.013.patch, YARN-7159.015.patch, > YARN-7159.016.patch, YARN-7159.017.patch, YARN-7159.018.patch, > YARN-7159.019.patch, YARN-7159.020.patch, YARN-7159.021.patch, > YARN-7159.022.patch, YARN-7159.023.patch > > > Currently resource conversion could happen in critical code path when > different unit is specified by client. This could impact performance and > throughput of RM a lot. We should do unit normalization when resource passed > to RM and avoid expensive unit conversion every time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7948) Enable fair scheduler to refresh maximum allocation for multiple resource types
[ https://issues.apache.org/jira/browse/YARN-7948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567118#comment-16567118 ] Szilard Nemeth commented on YARN-7948: -- Thanks [~haibochen]! > Enable fair scheduler to refresh maximum allocation for multiple resource > types > --- > > Key: YARN-7948 > URL: https://issues.apache.org/jira/browse/YARN-7948 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0 >Reporter: Yufei Gu >Assignee: Szilard Nemeth >Priority: Major > Fix For: 3.2.0 > > Attachments: YARN-7948.001.patch, YARN-7948.002.patch, > YARN-7948.003.patch, YARN-7948.004.patch > > > YARN-7738 did the same thing for CS. We need a fix for FS. We could fix it by > moving the refresh code from class CS to class AbstractYARNScheduler. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7948) Enable fair scheduler to refresh maximum allocation for multiple resource types
[ https://issues.apache.org/jira/browse/YARN-7948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567097#comment-16567097 ] Haibo Chen commented on YARN-7948: -- +1 on the 04 patch. Checking it in shortly. > Enable fair scheduler to refresh maximum allocation for multiple resource > types > --- > > Key: YARN-7948 > URL: https://issues.apache.org/jira/browse/YARN-7948 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0 >Reporter: Yufei Gu >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-7948.001.patch, YARN-7948.002.patch, > YARN-7948.003.patch, YARN-7948.004.patch > > > YARN-7738 did the same thing for CS. We need a fix for FS. We could fix it by > moving the refresh code from class CS to class AbstractYARNScheduler. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7948) Enable fair scheduler to refresh maximum allocation for multiple resource types
[ https://issues.apache.org/jira/browse/YARN-7948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7948: - Summary: Enable fair scheduler to refresh maximum allocation for multiple resource types (was: Enable refreshing maximum allocation for multiple resource types) > Enable fair scheduler to refresh maximum allocation for multiple resource > types > --- > > Key: YARN-7948 > URL: https://issues.apache.org/jira/browse/YARN-7948 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0 >Reporter: Yufei Gu >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-7948.001.patch, YARN-7948.002.patch, > YARN-7948.003.patch, YARN-7948.004.patch > > > YARN-7738 did the same thing for CS. We need a fix for FS. We could fix it by > moving the refresh code from class CS to class AbstractYARNScheduler. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8549) Adding a NoOp timeline writer and reader plugin classes for ATSv2
[ https://issues.apache.org/jira/browse/YARN-8549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabha Manepalli updated YARN-8549: --- Attachment: YARN-8549-branch-2.03.patch > Adding a NoOp timeline writer and reader plugin classes for ATSv2 > - > > Key: YARN-8549 > URL: https://issues.apache.org/jira/browse/YARN-8549 > Project: Hadoop YARN > Issue Type: Sub-task > Components: ATSv2, timelineclient, timelineserver >Reporter: Prabha Manepalli >Assignee: Prabha Manepalli >Priority: Minor > Attachments: YARN-8549-branch-2.03.patch, YARN-8549.v1.patch, > YARN-8549.v2.patch > > > Stub implementation for TimeLineReader and TimeLineWriter classes. > These are useful for functional testing of writer and reader path for ATSv2 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8448) AM HTTPS Support
[ https://issues.apache.org/jira/browse/YARN-8448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567078#comment-16567078 ] Robert Kanter commented on YARN-8448: - I'm not sure where Jenkins went, so I've just kicked one off manually > AM HTTPS Support > > > Key: YARN-8448 > URL: https://issues.apache.org/jira/browse/YARN-8448 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Robert Kanter >Assignee: Robert Kanter >Priority: Major > Attachments: YARN-8448.001.patch, YARN-8448.002.patch, > YARN-8448.003.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart
[
https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567076#comment-16567076
]
Haibo Chen commented on YARN-6966:
--
I have backported YARN-7542 to branch-2. Will kick off another Jenkins job
shortly.
> NodeManager metrics may return wrong negative values when NM restart
>
>
> Key: YARN-6966
> URL: https://issues.apache.org/jira/browse/YARN-6966
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Yang Wang
>Assignee: Szilard Nemeth
>Priority: Major
> Fix For: 3.2.0, 3.0.4, 3.1.2
>
> Attachments: YARN-6966-branch-2.001.patch,
> YARN-6966-branch-2.002.patch, YARN-6966-branch-2.002.patch,
> YARN-6966-branch-2.002.patch, YARN-6966-branch-3.0.0.001.patch,
> YARN-6966-branch-3.0.001.patch, YARN-6966.001.patch, YARN-6966.002.patch,
> YARN-6966.003.patch, YARN-6966.004.patch, YARN-6966.005.patch,
> YARN-6966.005.patch, YARN-6966.006.patch
>
>
> Just as YARN-6212. However, I think it is not a duplicate of YARN-3933.
> The primary cause of negative values is that metrics do not recover properly
> when NM restart.
> AllocatedContainers,ContainersLaunched,AllocatedGB,AvailableGB,AllocatedVCores,AvailableVCores
> in metrics also need to recover when NM restart.
> This should be done in ContainerManagerImpl#recoverContainer.
> The scenario could be reproduction by the following steps:
> # Make sure
> YarnConfiguration.NM_RECOVERY_ENABLED=true,YarnConfiguration.NM_RECOVERY_SUPERVISED=true
> in NM
> # Submit an application and keep running
> # Restart NM
> # Stop the application
> # Now you get the negative values
> {code}
> /jmx?qry=Hadoop:service=NodeManager,name=NodeManagerMetrics
> {code}
> {code}
> {
> name: "Hadoop:service=NodeManager,name=NodeManagerMetrics",
> modelerType: "NodeManagerMetrics",
> tag.Context: "yarn",
> tag.Hostname: "hadoop.com",
> ContainersLaunched: 0,
> ContainersCompleted: 0,
> ContainersFailed: 2,
> ContainersKilled: 0,
> ContainersIniting: 0,
> ContainersRunning: 0,
> AllocatedGB: 0,
> AllocatedContainers: -2,
> AvailableGB: 160,
> AllocatedVCores: -11,
> AvailableVCores: 3611,
> ContainerLaunchDurationNumOps: 2,
> ContainerLaunchDurationAvgTime: 6,
> BadLocalDirs: 0,
> BadLogDirs: 0,
> GoodLocalDirsDiskUtilizationPerc: 2,
> GoodLogDirsDiskUtilizationPerc: 2
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7542) Fix issue that causes some Running Opportunistic Containers to be recovered as PAUSED
[ https://issues.apache.org/jira/browse/YARN-7542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7542: - Fix Version/s: 2.10.0 > Fix issue that causes some Running Opportunistic Containers to be recovered > as PAUSED > - > > Key: YARN-7542 > URL: https://issues.apache.org/jira/browse/YARN-7542 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: Sampada Dehankar >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.1 > > Attachments: YARN-7542.001.patch > > > Steps to reproduce: > * Start YARN cluster - Enable Opportunistic containers and set NM queue > length to something > 10. Also Enable work preserving restart > * Start an MR job (without opportunistic containers) > * Kill the NM and restart it again. > * In the logs - it shows that some of the containers are in SUSPENDED state - > even though they are still running. > [~sampada15] / [~kartheek], can you take a look at this ? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7542) Fix issue that causes some Running Opportunistic Containers to be recovered as PAUSED
[ https://issues.apache.org/jira/browse/YARN-7542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567074#comment-16567074 ] Haibo Chen commented on YARN-7542: -- I have cherry-picked from branch-3.0 to branch-2, FYI. > Fix issue that causes some Running Opportunistic Containers to be recovered > as PAUSED > - > > Key: YARN-7542 > URL: https://issues.apache.org/jira/browse/YARN-7542 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Arun Suresh >Assignee: Sampada Dehankar >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.1 > > Attachments: YARN-7542.001.patch > > > Steps to reproduce: > * Start YARN cluster - Enable Opportunistic containers and set NM queue > length to something > 10. Also Enable work preserving restart > * Start an MR job (without opportunistic containers) > * Kill the NM and restart it again. > * In the logs - it shows that some of the containers are in SUSPENDED state - > even though they are still running. > [~sampada15] / [~kartheek], can you take a look at this ? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8612) Fix NM Collector Service Port issue in YarnConfiguration
[ https://issues.apache.org/jira/browse/YARN-8612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567068#comment-16567068 ] Prabha Manepalli edited comment on YARN-8612 at 8/2/18 5:00 PM: It is a config change and not a code change. There are no test cases associated with the change. was (Author: prabham): It is a config change and not a code change. There are no test cases associated with it. > Fix NM Collector Service Port issue in YarnConfiguration > > > Key: YARN-8612 > URL: https://issues.apache.org/jira/browse/YARN-8612 > Project: Hadoop YARN > Issue Type: Bug > Components: ATSv2 >Reporter: Prabha Manepalli >Assignee: Prabha Manepalli >Priority: Major > Attachments: YARN-8612.v1.patch > > > There is a typo in the existing YarnConfiguration which uses the > DEFAULT_NM_LOCALIZER_PORT as the default for NM Collector Service port. This > Jira aims to fix the typo. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7159) Normalize unit of resource objects in RM to avoid unit conversion in critical path
[ https://issues.apache.org/jira/browse/YARN-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-7159: - Summary: Normalize unit of resource objects in RM to avoid unit conversion in critical path (was: Normalize unit of resource objects in RM and avoid to do unit conversion in critical path) > Normalize unit of resource objects in RM to avoid unit conversion in critical > path > -- > > Key: YARN-7159 > URL: https://issues.apache.org/jira/browse/YARN-7159 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Wangda Tan >Assignee: Manikandan R >Priority: Critical > Attachments: YARN-7159.001.patch, YARN-7159.002.patch, > YARN-7159.003.patch, YARN-7159.004.patch, YARN-7159.005.patch, > YARN-7159.006.patch, YARN-7159.007.patch, YARN-7159.008.patch, > YARN-7159.009.patch, YARN-7159.010.patch, YARN-7159.011.patch, > YARN-7159.012.patch, YARN-7159.013.patch, YARN-7159.015.patch, > YARN-7159.016.patch, YARN-7159.017.patch, YARN-7159.018.patch, > YARN-7159.019.patch, YARN-7159.020.patch, YARN-7159.021.patch, > YARN-7159.022.patch, YARN-7159.023.patch > > > Currently resource conversion could happen in critical code path when > different unit is specified by client. This could impact performance and > throughput of RM a lot. We should do unit normalization when resource passed > to RM and avoid expensive unit conversion every time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8612) Fix NM Collector Service Port issue in YarnConfiguration
[ https://issues.apache.org/jira/browse/YARN-8612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567068#comment-16567068 ] Prabha Manepalli commented on YARN-8612: It is a config change and not a code change. There are no test cases associated with it. > Fix NM Collector Service Port issue in YarnConfiguration > > > Key: YARN-8612 > URL: https://issues.apache.org/jira/browse/YARN-8612 > Project: Hadoop YARN > Issue Type: Bug > Components: ATSv2 >Reporter: Prabha Manepalli >Assignee: Prabha Manepalli >Priority: Major > Attachments: YARN-8612.v1.patch > > > There is a typo in the existing YarnConfiguration which uses the > DEFAULT_NM_LOCALIZER_PORT as the default for NM Collector Service port. This > Jira aims to fix the typo. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4946) RM should not consider an application as COMPLETED when log aggregation is not in a terminal state
[
https://issues.apache.org/jira/browse/YARN-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567065#comment-16567065
]
genericqa commented on YARN-4946:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 57s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 26s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 69m 3s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}129m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart |
| | hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | YARN-4946 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12934092/YARN-4946.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 642dd3f93ff5 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 5033d7d |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/21487/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/21487/testReport/ |
| Max. process+thread count | 919 (vs. ulimit of 1) |
| modules | C:
[jira] [Commented] (YARN-7159) Normalize unit of resource objects in RM and avoid to do unit conversion in critical path
[ https://issues.apache.org/jira/browse/YARN-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567052#comment-16567052 ] Sunil Govindan commented on YARN-7159: -- Sorry. patch command works on this. Hence no worries. Thanks. > Normalize unit of resource objects in RM and avoid to do unit conversion in > critical path > - > > Key: YARN-7159 > URL: https://issues.apache.org/jira/browse/YARN-7159 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Wangda Tan >Assignee: Manikandan R >Priority: Critical > Attachments: YARN-7159.001.patch, YARN-7159.002.patch, > YARN-7159.003.patch, YARN-7159.004.patch, YARN-7159.005.patch, > YARN-7159.006.patch, YARN-7159.007.patch, YARN-7159.008.patch, > YARN-7159.009.patch, YARN-7159.010.patch, YARN-7159.011.patch, > YARN-7159.012.patch, YARN-7159.013.patch, YARN-7159.015.patch, > YARN-7159.016.patch, YARN-7159.017.patch, YARN-7159.018.patch, > YARN-7159.019.patch, YARN-7159.020.patch, YARN-7159.021.patch, > YARN-7159.022.patch, YARN-7159.023.patch > > > Currently resource conversion could happen in critical code path when > different unit is specified by client. This could impact performance and > throughput of RM a lot. We should do unit normalization when resource passed > to RM and avoid expensive unit conversion every time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7159) Normalize unit of resource objects in RM and avoid to do unit conversion in critical path
[ https://issues.apache.org/jira/browse/YARN-7159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567034#comment-16567034 ] Sunil Govindan commented on YARN-7159: -- [~maniraj...@gmail.com] I think patch needs a rebase. Not applying on latest trunk. cud u pls help. > Normalize unit of resource objects in RM and avoid to do unit conversion in > critical path > - > > Key: YARN-7159 > URL: https://issues.apache.org/jira/browse/YARN-7159 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Reporter: Wangda Tan >Assignee: Manikandan R >Priority: Critical > Attachments: YARN-7159.001.patch, YARN-7159.002.patch, > YARN-7159.003.patch, YARN-7159.004.patch, YARN-7159.005.patch, > YARN-7159.006.patch, YARN-7159.007.patch, YARN-7159.008.patch, > YARN-7159.009.patch, YARN-7159.010.patch, YARN-7159.011.patch, > YARN-7159.012.patch, YARN-7159.013.patch, YARN-7159.015.patch, > YARN-7159.016.patch, YARN-7159.017.patch, YARN-7159.018.patch, > YARN-7159.019.patch, YARN-7159.020.patch, YARN-7159.021.patch, > YARN-7159.022.patch, YARN-7159.023.patch > > > Currently resource conversion could happen in critical code path when > different unit is specified by client. This could impact performance and > throughput of RM a lot. We should do unit normalization when resource passed > to RM and avoid expensive unit conversion every time. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7948) Enable refreshing maximum allocation for multiple resource types
[
https://issues.apache.org/jira/browse/YARN-7948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567025#comment-16567025
]
genericqa commented on YARN-7948:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
38s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 37m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 29s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 34s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 73m
59s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}142m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | YARN-7948 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12934085/YARN-7948.004.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 38ebd394852c 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14
08:53:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / e83719c |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/21485/testReport/ |
| Max. process+thread count | 951 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/21485/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Enable refreshing maximum allocation for multiple
[jira] [Commented] (YARN-8263) DockerClient still touches hadoop.tmp.dir
[ https://issues.apache.org/jira/browse/YARN-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567006#comment-16567006 ] Hudson commented on YARN-8263: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14698 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14698/]) YARN-8263. DockerClient still touches hadoop.tmp.dir. Contributed by (jlowe: rev 7526815e3234ca352854ecfb142a13f1a188d5bd) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/DockerCommandExecutor.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/DockerInspectCommand.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/TestDockerCommandExecutor.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/DockerLinuxContainerRuntime.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/DockerRmCommand.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/TestDockerClient.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/DockerClient.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/docker/DockerCommand.java > DockerClient still touches hadoop.tmp.dir > - > > Key: YARN-8263 > URL: https://issues.apache.org/jira/browse/YARN-8263 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.1.1 >Reporter: Jason Lowe >Assignee: Craig Condit >Priority: Minor > Labels: Docker > Fix For: 3.2.0, 3.1.2 > > Attachments: YARN-8263.001.patch, YARN-8263.002.patch, > YARN-8263.003.patch > > > The DockerClient constructor fails if hadoop.tmp.dir is not set and proceeds > to create a directory there. After YARN-8064 there's no longer a need to > touch the temporary directory. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8613) Old RM UI shows wrong vcores total value
[ https://issues.apache.org/jira/browse/YARN-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566557#comment-16566557 ] Bibin A Chundatt edited comment on YARN-8613 at 8/2/18 4:01 PM: [~sunilg] Yes . Looks like queue metrics issue . IIRC the Cluster Resource is shown from QueueMetrics , and reserved cores created issue earlier . was (Author: bibinchundatt): [~sunilg] Yes . Looks like queue metrics issue . IIRC the Cluster Resource is shown from QueueMetrics , and reservation used on create issue earlier . > Old RM UI shows wrong vcores total value > > > Key: YARN-8613 > URL: https://issues.apache.org/jira/browse/YARN-8613 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akhil PB >Priority: Major > Attachments: Screen Shot 2018-08-02 at 12.12.41 PM.png, Screen Shot > 2018-08-02 at 12.16.53 PM.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8318) [UI2] IP address in component page shows N/A
[
https://issues.apache.org/jira/browse/YARN-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566961#comment-16566961
]
genericqa commented on YARN-8318:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
39m 6s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 32s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 52m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | YARN-8318 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12924021/YARN-8318.001.patch |
| Optional Tests | asflicense shadedclient |
| uname | Linux 2082537ccc5c 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / e83719c |
| maven | version: Apache Maven 3.3.9 |
| Max. process+thread count | 289 (vs. ulimit of 1) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/21486/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> [UI2] IP address in component page shows N/A
>
>
> Key: YARN-8318
> URL: https://issues.apache.org/jira/browse/YARN-8318
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn-ui-v2
>Affects Versions: 3.1.0
>Reporter: Yesha Vora
>Assignee: Yesha Vora
>Priority: Major
> Fix For: 3.2.0, 3.1.2
>
> Attachments: YARN-8318.001.patch
>
>
> Component page shows IP address value as N/A. It should print IP address of
> docker container.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-4946) RM should not consider an application as COMPLETED when log aggregation is not in a terminal state
[
https://issues.apache.org/jira/browse/YARN-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566890#comment-16566890
]
Szilard Nemeth edited comment on YARN-4946 at 8/2/18 3:27 PM:
--
DEV NOTES:
The initial implementation could have looked like this:
The very first line of transition should be to check whether log aggregation is
finished.
If it doesn't, don't do anything and break from the method.
To make sure apps become completed if log aggregation is finished, the
APP_COMPLETED event need to be dispatched when log aggregation finishes.
In my understanding, this is the sequence of events:
1. RM receives NM heartbeat in ResourceTrackerService.nodeUpdate
2. An RmNodeEvent is created with type STATUS_UPDATE
3. RmNodeImpl.StatusUpdateWhenHealthyTransition.transition handles the node
status update
4. If there are any log aggregation reports then
{{RmNode#handleLogAggregationStatus}} is called
5. This ultimately calls rmApp.aggregateLogReport
In rmApp.aggregateLogReport, I needed to check whether log aggregation finished
and then send the APP_COMPLETED event.
An issue with this approach:
If a {{FinalTransition}} runs because of the app got killed, finished or
rejected, e.g. RMAppImpl goes from the RUNNING to FINISHED state
(RMAppEventType.ATTEMPT_FINISHED), no matter what happens in
{{FinalTransition}}, the app will reach a terminal state (FINISHED in this
case).
If I would use a break statement as described above, the app would be in a
FINISHED state which is not right as the rest of the code in the transition
could not run again.
So with my implementation, all the code in {{FinalTransition}} runs like as
before and if log aggregation is not finished yet, I don't send the
APP_COMPLETED event to the {{RMAppManager}}.
When the log aggregation is finished for an app,
{{RMAppImpl#aggregateLogReport}} will be called.
In this method, I added a piece of code that sends the APP_COMPLETED event to
the {{RMAppManager}} if the application is in a final state.
was (Author: snemeth):
DEV NOTES:
The initial implementation could have looked it like this:
The very first line of transition should be to check whether log aggregation is
finished.
If it doesn't, don't do anything and break from the method.
To make sure apps become completed if log aggregation is finished, the
APP_COMPLETED event need to be dispatched when log aggregation finishes.
In my understanding, this is the sequence of events:
1. RM receives NM heartbeat in ResourceTrackerService.nodeUpdate
2. An RmNodeEvent is created with type STATUS_UPDATE
3. RmNodeImpl.StatusUpdateWhenHealthyTransition.transition handles the node
status update
4. If there are any log aggregation reports then
{{RmNode#handleLogAggregationStatus}} is called
5. This ultimately calls rmApp.aggregateLogReport
In rmApp.aggregateLogReport, I needed to check whether log aggregation finished
and then send the APP_COMPLETED event.
An issue with this approach:
If a {{FinalTransition}} runs because of the app got killed, finished or
rejected, e.g. RMAppImpl goes from the RUNNING to FINISHED state
(RMAppEventType.ATTEMPT_FINISHED), no matter what happens in
{{FinalTransition}}, the app will reach a terminal state (FINISHED in this
case).
If I would use a break statement as described above, the app would be in a
FINISHED state which is not right as the rest of the code in the transition
could not run again.
So with my implementation, all the code in {{FinalTransition}} runs like as
before and if log aggregation is not finished yet, I don't send the
APP_COMPLETED event to the {{RMAppManager}}.
When the log aggregation is finished for an app,
{{RMAppImpl#aggregateLogReport}} will be called.
In this method, I added a piece of code that sends the APP_COMPLETED event to
the {{RMAppManager}} if the application is in a final state.
> RM should not consider an application as COMPLETED when log aggregation is
> not in a terminal state
> --
>
> Key: YARN-4946
> URL: https://issues.apache.org/jira/browse/YARN-4946
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: log-aggregation
>Affects Versions: 2.8.0
>Reporter: Robert Kanter
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-4946.001.patch, YARN-4946.002.patch
>
>
> MAPREDUCE-6415 added a tool that combines the aggregated log files for each
> Yarn App into a HAR file. When run, it seeds the list by looking at the
> aggregated logs directory, and then filters out ineligible apps. One of the
> criteria involves checking with the RM that an Application's log aggregation
> status is not still running and has not failed. When the RM "forgets" about
> an older completed
[jira] [Comment Edited] (YARN-4946) RM should not consider an application as COMPLETED when log aggregation is not in a terminal state
[
https://issues.apache.org/jira/browse/YARN-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566890#comment-16566890
]
Szilard Nemeth edited comment on YARN-4946 at 8/2/18 3:26 PM:
--
DEV NOTES:
An initial implementation could have looked it like this:
The very first line of transition should be to check whether log aggregation is
finished.
If it doesn't, don't do anything and break from the method.
To make sure apps become completed if log aggregation is finished, the
APP_COMPLETED event need to be dispatched when log aggregation finishes.
In my understanding, this is the sequence of events:
1. RM receives NM heartbeat in ResourceTrackerService.nodeUpdate
2. An RmNodeEvent is created with type STATUS_UPDATE
3. RmNodeImpl.StatusUpdateWhenHealthyTransition.transition handles the node
status update
4. If there are any log aggregation reports then
{{RmNode#handleLogAggregationStatus}} is called
5. This ultimately calls rmApp.aggregateLogReport
In rmApp.aggregateLogReport, I needed to check whether log aggregation finished
and then send the APP_COMPLETED event.
An issue with this approach:
If a {{FinalTransition}} runs because of the app got killed, finished or
rejected, e.g. RMAppImpl goes from the RUNNING to FINISHED state
(RMAppEventType.ATTEMPT_FINISHED), no matter what happens in
{{FinalTransition}}, the app will reach a terminal state (FINISHED in this
case).
If I would use a break statement as described above, the app would be in a
FINISHED state which is not right as the rest of the code in the transition
could not run again.
So with my implementation, all the code in {{FinalTransition}} runs like as
before and if log aggregation is not finished yet, I don't send the
APP_COMPLETED event to the {{RMAppManager}}.
When the log aggregation is finished for an app,
{{RMAppImpl.aggregateLogReport}} will be called.
In this method, I added a piece of code that sends the APP_COMPLETED event to
the {{RMAppManager}} if the application is in a final state.
was (Author: snemeth):
DEV NOTES:
An initial implementation could have looked it like this:
The very first line of transition should be to check whether log aggregation is
finished.
If it doesn't, don't do anything and break from the method.
To make sure apps become completed if log aggregation is finished, the
APP_COMPLETED event need to be dispatched when log aggregation finishes.
In my understanding, this is the sequence of events:
1. RM receives NM heartbeat in ResourceTrackerService.nodeUpdate
2. An RmNodeEvent is created with type STATUS_UPDATE
3. RmNodeImpl.StatusUpdateWhenHealthyTransition.transition handles the node
status update
4. If there is any log aggregation reports then
RmNode.handleLogAggregationStatus is called
5. This ultimately calls rmApp.aggregateLogReport
In rmApp.aggregateLogReport, I needed to check whether log aggregation finished
and then send the APP_COMPLETED event.
An issue with this approach:
If a {{FinalTransition}} runs because of the app got killed, finished or
rejected, e.g. RMAppImpl goes from the RUNNING to FINISHED state
(RMAppEventType.ATTEMPT_FINISHED), no matter what happens in
{{FinalTransition}}, the app will reach a terminal state (FINISHED in this
case).
If I would use a break statement as described above, the app would be in a
FINISHED state which is not right as the rest of the code in the transition
could not run again.
So with my implementation, I run all the code in {{FinalTransition}} as before
and if log aggregation is not finished yet, I don't send the APP_COMPLETED
event to the {{RMAppManager}}.
When the log aggregation is finished for an app,
{{RMAppImpl.aggregateLogReport}} will be called.
In this method, I added a piece of code that sends the APP_COMPLETED event to
the {{RMAppManager}} if the application is in a final state.
> RM should not consider an application as COMPLETED when log aggregation is
> not in a terminal state
> --
>
> Key: YARN-4946
> URL: https://issues.apache.org/jira/browse/YARN-4946
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: log-aggregation
>Affects Versions: 2.8.0
>Reporter: Robert Kanter
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-4946.001.patch, YARN-4946.002.patch
>
>
> MAPREDUCE-6415 added a tool that combines the aggregated log files for each
> Yarn App into a HAR file. When run, it seeds the list by looking at the
> aggregated logs directory, and then filters out ineligible apps. One of the
> criteria involves checking with the RM that an Application's log aggregation
> status is not still running and has not failed. When the RM "forgets" about
> an older completed Application (e.g.
[jira] [Comment Edited] (YARN-4946) RM should not consider an application as COMPLETED when log aggregation is not in a terminal state
[
https://issues.apache.org/jira/browse/YARN-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566890#comment-16566890
]
Szilard Nemeth edited comment on YARN-4946 at 8/2/18 3:26 PM:
--
DEV NOTES:
The initial implementation could have looked it like this:
The very first line of transition should be to check whether log aggregation is
finished.
If it doesn't, don't do anything and break from the method.
To make sure apps become completed if log aggregation is finished, the
APP_COMPLETED event need to be dispatched when log aggregation finishes.
In my understanding, this is the sequence of events:
1. RM receives NM heartbeat in ResourceTrackerService.nodeUpdate
2. An RmNodeEvent is created with type STATUS_UPDATE
3. RmNodeImpl.StatusUpdateWhenHealthyTransition.transition handles the node
status update
4. If there are any log aggregation reports then
{{RmNode#handleLogAggregationStatus}} is called
5. This ultimately calls rmApp.aggregateLogReport
In rmApp.aggregateLogReport, I needed to check whether log aggregation finished
and then send the APP_COMPLETED event.
An issue with this approach:
If a {{FinalTransition}} runs because of the app got killed, finished or
rejected, e.g. RMAppImpl goes from the RUNNING to FINISHED state
(RMAppEventType.ATTEMPT_FINISHED), no matter what happens in
{{FinalTransition}}, the app will reach a terminal state (FINISHED in this
case).
If I would use a break statement as described above, the app would be in a
FINISHED state which is not right as the rest of the code in the transition
could not run again.
So with my implementation, all the code in {{FinalTransition}} runs like as
before and if log aggregation is not finished yet, I don't send the
APP_COMPLETED event to the {{RMAppManager}}.
When the log aggregation is finished for an app,
{{RMAppImpl#aggregateLogReport}} will be called.
In this method, I added a piece of code that sends the APP_COMPLETED event to
the {{RMAppManager}} if the application is in a final state.
was (Author: snemeth):
DEV NOTES:
An initial implementation could have looked it like this:
The very first line of transition should be to check whether log aggregation is
finished.
If it doesn't, don't do anything and break from the method.
To make sure apps become completed if log aggregation is finished, the
APP_COMPLETED event need to be dispatched when log aggregation finishes.
In my understanding, this is the sequence of events:
1. RM receives NM heartbeat in ResourceTrackerService.nodeUpdate
2. An RmNodeEvent is created with type STATUS_UPDATE
3. RmNodeImpl.StatusUpdateWhenHealthyTransition.transition handles the node
status update
4. If there are any log aggregation reports then
{{RmNode#handleLogAggregationStatus}} is called
5. This ultimately calls rmApp.aggregateLogReport
In rmApp.aggregateLogReport, I needed to check whether log aggregation finished
and then send the APP_COMPLETED event.
An issue with this approach:
If a {{FinalTransition}} runs because of the app got killed, finished or
rejected, e.g. RMAppImpl goes from the RUNNING to FINISHED state
(RMAppEventType.ATTEMPT_FINISHED), no matter what happens in
{{FinalTransition}}, the app will reach a terminal state (FINISHED in this
case).
If I would use a break statement as described above, the app would be in a
FINISHED state which is not right as the rest of the code in the transition
could not run again.
So with my implementation, all the code in {{FinalTransition}} runs like as
before and if log aggregation is not finished yet, I don't send the
APP_COMPLETED event to the {{RMAppManager}}.
When the log aggregation is finished for an app,
{{RMAppImpl#aggregateLogReport}} will be called.
In this method, I added a piece of code that sends the APP_COMPLETED event to
the {{RMAppManager}} if the application is in a final state.
> RM should not consider an application as COMPLETED when log aggregation is
> not in a terminal state
> --
>
> Key: YARN-4946
> URL: https://issues.apache.org/jira/browse/YARN-4946
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: log-aggregation
>Affects Versions: 2.8.0
>Reporter: Robert Kanter
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-4946.001.patch, YARN-4946.002.patch
>
>
> MAPREDUCE-6415 added a tool that combines the aggregated log files for each
> Yarn App into a HAR file. When run, it seeds the list by looking at the
> aggregated logs directory, and then filters out ineligible apps. One of the
> criteria involves checking with the RM that an Application's log aggregation
> status is not still running and has not failed. When the RM "forgets" about
> an older completed
[jira] [Comment Edited] (YARN-4946) RM should not consider an application as COMPLETED when log aggregation is not in a terminal state
[
https://issues.apache.org/jira/browse/YARN-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566890#comment-16566890
]
Szilard Nemeth edited comment on YARN-4946 at 8/2/18 3:26 PM:
--
DEV NOTES:
An initial implementation could have looked it like this:
The very first line of transition should be to check whether log aggregation is
finished.
If it doesn't, don't do anything and break from the method.
To make sure apps become completed if log aggregation is finished, the
APP_COMPLETED event need to be dispatched when log aggregation finishes.
In my understanding, this is the sequence of events:
1. RM receives NM heartbeat in ResourceTrackerService.nodeUpdate
2. An RmNodeEvent is created with type STATUS_UPDATE
3. RmNodeImpl.StatusUpdateWhenHealthyTransition.transition handles the node
status update
4. If there are any log aggregation reports then
{{RmNode#handleLogAggregationStatus}} is called
5. This ultimately calls rmApp.aggregateLogReport
In rmApp.aggregateLogReport, I needed to check whether log aggregation finished
and then send the APP_COMPLETED event.
An issue with this approach:
If a {{FinalTransition}} runs because of the app got killed, finished or
rejected, e.g. RMAppImpl goes from the RUNNING to FINISHED state
(RMAppEventType.ATTEMPT_FINISHED), no matter what happens in
{{FinalTransition}}, the app will reach a terminal state (FINISHED in this
case).
If I would use a break statement as described above, the app would be in a
FINISHED state which is not right as the rest of the code in the transition
could not run again.
So with my implementation, all the code in {{FinalTransition}} runs like as
before and if log aggregation is not finished yet, I don't send the
APP_COMPLETED event to the {{RMAppManager}}.
When the log aggregation is finished for an app,
{{RMAppImpl#aggregateLogReport}} will be called.
In this method, I added a piece of code that sends the APP_COMPLETED event to
the {{RMAppManager}} if the application is in a final state.
was (Author: snemeth):
DEV NOTES:
An initial implementation could have looked it like this:
The very first line of transition should be to check whether log aggregation is
finished.
If it doesn't, don't do anything and break from the method.
To make sure apps become completed if log aggregation is finished, the
APP_COMPLETED event need to be dispatched when log aggregation finishes.
In my understanding, this is the sequence of events:
1. RM receives NM heartbeat in ResourceTrackerService.nodeUpdate
2. An RmNodeEvent is created with type STATUS_UPDATE
3. RmNodeImpl.StatusUpdateWhenHealthyTransition.transition handles the node
status update
4. If there are any log aggregation reports then
{{RmNode#handleLogAggregationStatus}} is called
5. This ultimately calls rmApp.aggregateLogReport
In rmApp.aggregateLogReport, I needed to check whether log aggregation finished
and then send the APP_COMPLETED event.
An issue with this approach:
If a {{FinalTransition}} runs because of the app got killed, finished or
rejected, e.g. RMAppImpl goes from the RUNNING to FINISHED state
(RMAppEventType.ATTEMPT_FINISHED), no matter what happens in
{{FinalTransition}}, the app will reach a terminal state (FINISHED in this
case).
If I would use a break statement as described above, the app would be in a
FINISHED state which is not right as the rest of the code in the transition
could not run again.
So with my implementation, all the code in {{FinalTransition}} runs like as
before and if log aggregation is not finished yet, I don't send the
APP_COMPLETED event to the {{RMAppManager}}.
When the log aggregation is finished for an app,
{{RMAppImpl.aggregateLogReport}} will be called.
In this method, I added a piece of code that sends the APP_COMPLETED event to
the {{RMAppManager}} if the application is in a final state.
> RM should not consider an application as COMPLETED when log aggregation is
> not in a terminal state
> --
>
> Key: YARN-4946
> URL: https://issues.apache.org/jira/browse/YARN-4946
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: log-aggregation
>Affects Versions: 2.8.0
>Reporter: Robert Kanter
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-4946.001.patch, YARN-4946.002.patch
>
>
> MAPREDUCE-6415 added a tool that combines the aggregated log files for each
> Yarn App into a HAR file. When run, it seeds the list by looking at the
> aggregated logs directory, and then filters out ineligible apps. One of the
> criteria involves checking with the RM that an Application's log aggregation
> status is not still running and has not failed. When the RM "forgets" about
> an older completed
[jira] [Commented] (YARN-8263) DockerClient still touches hadoop.tmp.dir
[ https://issues.apache.org/jira/browse/YARN-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566936#comment-16566936 ] Jason Lowe commented on YARN-8263: -- Thanks for updating the patch! +1 lgtm. There are now some unused Configuration imports in a few files, but I can clean those up in the commit. > DockerClient still touches hadoop.tmp.dir > - > > Key: YARN-8263 > URL: https://issues.apache.org/jira/browse/YARN-8263 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.1.1 >Reporter: Jason Lowe >Assignee: Craig Condit >Priority: Minor > Labels: Docker > Attachments: YARN-8263.001.patch, YARN-8263.002.patch, > YARN-8263.003.patch > > > The DockerClient constructor fails if hadoop.tmp.dir is not set and proceeds > to create a directory there. After YARN-8064 there's no longer a need to > touch the temporary directory. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8616) System.currentTimeMillis() used in RMAppImpl, instead of getting value from systemClock
Szilard Nemeth created YARN-8616: Summary: System.currentTimeMillis() used in RMAppImpl, instead of getting value from systemClock Key: YARN-8616 URL: https://issues.apache.org/jira/browse/YARN-8616 Project: Hadoop YARN Issue Type: Improvement Reporter: Szilard Nemeth Assignee: Szilard Nemeth -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8318) [UI2] IP address in component page shows N/A
[ https://issues.apache.org/jira/browse/YARN-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566926#comment-16566926 ] Hudson commented on YARN-8318: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14697 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14697/]) YARN-8318. [UI2] IP address in component page shows N/A. Contributed by (sunilg: rev 5033d7da8f6f703d8774492c42e31e9b9cb692a5) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/templates/yarn-component-instance/info.hbs > [UI2] IP address in component page shows N/A > > > Key: YARN-8318 > URL: https://issues.apache.org/jira/browse/YARN-8318 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.0 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Fix For: 3.2.0, 3.1.2 > > Attachments: YARN-8318.001.patch > > > Component page shows IP address value as N/A. It should print IP address of > docker container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4946) RM should not consider an application as COMPLETED when log aggregation is not in a terminal state
[ https://issues.apache.org/jira/browse/YARN-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566903#comment-16566903 ] Szilard Nemeth commented on YARN-4946: -- Hi [~csingh]! I think the log aggregation status is only available from RM, but I would like to ask [~rkanter] to confirm this. Anyway, I think introducing a new dependency on AHS or ATS for HadoopArchiveLogs would complicate things unnecessarily. > RM should not consider an application as COMPLETED when log aggregation is > not in a terminal state > -- > > Key: YARN-4946 > URL: https://issues.apache.org/jira/browse/YARN-4946 > Project: Hadoop YARN > Issue Type: Improvement > Components: log-aggregation >Affects Versions: 2.8.0 >Reporter: Robert Kanter >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-4946.001.patch, YARN-4946.002.patch > > > MAPREDUCE-6415 added a tool that combines the aggregated log files for each > Yarn App into a HAR file. When run, it seeds the list by looking at the > aggregated logs directory, and then filters out ineligible apps. One of the > criteria involves checking with the RM that an Application's log aggregation > status is not still running and has not failed. When the RM "forgets" about > an older completed Application (e.g. RM failover, enough time has passed, > etc), the tool won't find the Application in the RM and will just assume that > its log aggregation succeeded, even if it actually failed or is still running. > We can solve this problem by doing the following: > The RM should not consider an app to be fully completed (and thus removed > from its history) until the aggregation status has reached a terminal state > (e.g. SUCCEEDED, FAILED, TIME_OUT). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4946) RM should not consider an application as COMPLETED when log aggregation is not in a terminal state
[ https://issues.apache.org/jira/browse/YARN-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566896#comment-16566896 ] Szilard Nemeth commented on YARN-4946: -- uploaded patch002 to fix whitespace issues. The unit test failure is unrelated (known flakey) > RM should not consider an application as COMPLETED when log aggregation is > not in a terminal state > -- > > Key: YARN-4946 > URL: https://issues.apache.org/jira/browse/YARN-4946 > Project: Hadoop YARN > Issue Type: Improvement > Components: log-aggregation >Affects Versions: 2.8.0 >Reporter: Robert Kanter >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-4946.001.patch, YARN-4946.002.patch > > > MAPREDUCE-6415 added a tool that combines the aggregated log files for each > Yarn App into a HAR file. When run, it seeds the list by looking at the > aggregated logs directory, and then filters out ineligible apps. One of the > criteria involves checking with the RM that an Application's log aggregation > status is not still running and has not failed. When the RM "forgets" about > an older completed Application (e.g. RM failover, enough time has passed, > etc), the tool won't find the Application in the RM and will just assume that > its log aggregation succeeded, even if it actually failed or is still running. > We can solve this problem by doing the following: > The RM should not consider an app to be fully completed (and thus removed > from its history) until the aggregation status has reached a terminal state > (e.g. SUCCEEDED, FAILED, TIME_OUT). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4946) RM should not consider an application as COMPLETED when log aggregation is not in a terminal state
[ https://issues.apache.org/jira/browse/YARN-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-4946: - Attachment: YARN-4946.002.patch > RM should not consider an application as COMPLETED when log aggregation is > not in a terminal state > -- > > Key: YARN-4946 > URL: https://issues.apache.org/jira/browse/YARN-4946 > Project: Hadoop YARN > Issue Type: Improvement > Components: log-aggregation >Affects Versions: 2.8.0 >Reporter: Robert Kanter >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-4946.001.patch, YARN-4946.002.patch > > > MAPREDUCE-6415 added a tool that combines the aggregated log files for each > Yarn App into a HAR file. When run, it seeds the list by looking at the > aggregated logs directory, and then filters out ineligible apps. One of the > criteria involves checking with the RM that an Application's log aggregation > status is not still running and has not failed. When the RM "forgets" about > an older completed Application (e.g. RM failover, enough time has passed, > etc), the tool won't find the Application in the RM and will just assume that > its log aggregation succeeded, even if it actually failed or is still running. > We can solve this problem by doing the following: > The RM should not consider an app to be fully completed (and thus removed > from its history) until the aggregation status has reached a terminal state > (e.g. SUCCEEDED, FAILED, TIME_OUT). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4946) RM should not consider an application as COMPLETED when log aggregation is not in a terminal state
[
https://issues.apache.org/jira/browse/YARN-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566890#comment-16566890
]
Szilard Nemeth commented on YARN-4946:
--
DEV NOTES:
An initial implementation could have looked it like this:
The very first line of transition should be to check whether log aggregation is
finished.
If it doesn't, don't do anything and break from the method.
To make sure apps become completed if log aggregation is finished, the
APP_COMPLETED event need to be dispatched when log aggregation finishes.
In my understanding, this is the sequence of events:
1. RM receives NM heartbeat in ResourceTrackerService.nodeUpdate
2. An RmNodeEvent is created with type STATUS_UPDATE
3. RmNodeImpl.StatusUpdateWhenHealthyTransition.transition handles the node
status update
4. If there is any log aggregation reports then
RmNode.handleLogAggregationStatus is called
5. This ultimately calls rmApp.aggregateLogReport
In rmApp.aggregateLogReport, I needed to check whether log aggregation finished
and then send the APP_COMPLETED event.
An issue with this approach:
If a {{FinalTransition}} runs because of the app got killed, finished or
rejected, e.g. RMAppImpl goes from the RUNNING to FINISHED state
(RMAppEventType.ATTEMPT_FINISHED), no matter what happens in
{{FinalTransition}}, the app will reach a terminal state (FINISHED in this
case).
If I would use a break statement as described above, the app would be in a
FINISHED state which is not right as the rest of the code in the transition
could not run again.
So with my implementation, I run all the code in {{FinalTransition}} as before
and if log aggregation is not finished yet, I don't send the APP_COMPLETED
event to the {{RMAppManager}}.
When the log aggregation is finished for an app,
{{RMAppImpl.aggregateLogReport}} will be called.
In this method, I added a piece of code that sends the APP_COMPLETED event to
the {{RMAppManager}} if the application is in a final state.
> RM should not consider an application as COMPLETED when log aggregation is
> not in a terminal state
> --
>
> Key: YARN-4946
> URL: https://issues.apache.org/jira/browse/YARN-4946
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: log-aggregation
>Affects Versions: 2.8.0
>Reporter: Robert Kanter
>Assignee: Szilard Nemeth
>Priority: Major
> Attachments: YARN-4946.001.patch
>
>
> MAPREDUCE-6415 added a tool that combines the aggregated log files for each
> Yarn App into a HAR file. When run, it seeds the list by looking at the
> aggregated logs directory, and then filters out ineligible apps. One of the
> criteria involves checking with the RM that an Application's log aggregation
> status is not still running and has not failed. When the RM "forgets" about
> an older completed Application (e.g. RM failover, enough time has passed,
> etc), the tool won't find the Application in the RM and will just assume that
> its log aggregation succeeded, even if it actually failed or is still running.
> We can solve this problem by doing the following:
> The RM should not consider an app to be fully completed (and thus removed
> from its history) until the aggregation status has reached a terminal state
> (e.g. SUCCEEDED, FAILED, TIME_OUT).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8318) [UI2] IP address in component page shows N/A
[ https://issues.apache.org/jira/browse/YARN-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8318: - Summary: [UI2] IP address in component page shows N/A (was: IP address in component page shows N/A) > [UI2] IP address in component page shows N/A > > > Key: YARN-8318 > URL: https://issues.apache.org/jira/browse/YARN-8318 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.0 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8318.001.patch > > > Component page shows IP address value as N/A. It should print IP address of > docker container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8318) IP address in component page shows N/A
[ https://issues.apache.org/jira/browse/YARN-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566875#comment-16566875 ] Sunil Govindan commented on YARN-8318: -- This patch looks good and straightforward. Some how fell out my radar. Committing shortly. > IP address in component page shows N/A > -- > > Key: YARN-8318 > URL: https://issues.apache.org/jira/browse/YARN-8318 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Affects Versions: 3.1.0 >Reporter: Yesha Vora >Assignee: Yesha Vora >Priority: Major > Attachments: YARN-8318.001.patch > > > Component page shows IP address value as N/A. It should print IP address of > docker container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-7948) Enable refreshing maximum allocation for multiple resource types
[ https://issues.apache.org/jira/browse/YARN-7948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-7948: - Attachment: YARN-7948.004.patch > Enable refreshing maximum allocation for multiple resource types > > > Key: YARN-7948 > URL: https://issues.apache.org/jira/browse/YARN-7948 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0 >Reporter: Yufei Gu >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-7948.001.patch, YARN-7948.002.patch, > YARN-7948.003.patch, YARN-7948.004.patch > > > YARN-7738 did the same thing for CS. We need a fix for FS. We could fix it by > moving the refresh code from class CS to class AbstractYARNScheduler. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7948) Enable refreshing maximum allocation for multiple resource types
[ https://issues.apache.org/jira/browse/YARN-7948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566839#comment-16566839 ] Szilard Nemeth commented on YARN-7948: -- Hi [~haibochen]! You are right about the imports, not sure why my IDE haven't showed them unused when I checked it last time. Sorry about that. Reformatted the code changes to use 4 spaces as continuation indentation. Please check the latest patch! Thanks! > Enable refreshing maximum allocation for multiple resource types > > > Key: YARN-7948 > URL: https://issues.apache.org/jira/browse/YARN-7948 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 3.0.0 >Reporter: Yufei Gu >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-7948.001.patch, YARN-7948.002.patch, > YARN-7948.003.patch, YARN-7948.004.patch > > > YARN-7738 did the same thing for CS. We need a fix for FS. We could fix it by > moving the refresh code from class CS to class AbstractYARNScheduler. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8559) Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint
[ https://issues.apache.org/jira/browse/YARN-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566711#comment-16566711 ] Sunil Govindan commented on YARN-8559: -- Thanks [~cheersyang]. Looks good. Only concern was that because we use initWritableEndPoint is called from GET api. However as [~leftnoteasy] mentioned, its better to protect these for admin for now. Thanks for adding some more comments there. If no objections, I can help to commit the same tomorrow. > Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint > > > Key: YARN-8559 > URL: https://issues.apache.org/jira/browse/YARN-8559 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Reporter: Anna Savarin >Assignee: Weiwei Yang >Priority: Major > Attachments: YARN-8559.001.patch, YARN-8559.002.patch, > YARN-8559.003.patch, YARN-8559.004.patch > > > All Hadoop services provide a set of common endpoints (/stacks, /logLevel, > /metrics, /jmx, /conf). In the case of the Resource Manager, part of the > configuration comes from the scheduler being used. Currently, these > configuration key/values are not exposed through the /conf endpoint, thereby > revealing an incomplete configuration picture. > Make an improvement and expose the scheduling configuration info through the > RM's /conf endpoint. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6966) NodeManager metrics may return wrong negative values when NM restart
[
https://issues.apache.org/jira/browse/YARN-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566683#comment-16566683
]
Szilard Nemeth commented on YARN-6966:
--
Hi [~haibochen]!
I found out what is causing the test to fail.
There is a missing backport on branch-2: YARN-7542
In {{RecoveredContainerLaunch}}, in {{call}} the {{ContainerEventType}} being
sent is {{PAUSED}} instead of {{CONTAINER_LAUNCHED}}. This ultimately puts the
container to PAUSED state instead of RUNNING.
When the container becomes RUNNING, the running container metrics is increased.
Could you please do the backport of YARN-7452 to branch-2?
Thanks!
> NodeManager metrics may return wrong negative values when NM restart
>
>
> Key: YARN-6966
> URL: https://issues.apache.org/jira/browse/YARN-6966
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Yang Wang
>Assignee: Szilard Nemeth
>Priority: Major
> Fix For: 3.2.0, 3.0.4, 3.1.2
>
> Attachments: YARN-6966-branch-2.001.patch,
> YARN-6966-branch-2.002.patch, YARN-6966-branch-2.002.patch,
> YARN-6966-branch-2.002.patch, YARN-6966-branch-3.0.0.001.patch,
> YARN-6966-branch-3.0.001.patch, YARN-6966.001.patch, YARN-6966.002.patch,
> YARN-6966.003.patch, YARN-6966.004.patch, YARN-6966.005.patch,
> YARN-6966.005.patch, YARN-6966.006.patch
>
>
> Just as YARN-6212. However, I think it is not a duplicate of YARN-3933.
> The primary cause of negative values is that metrics do not recover properly
> when NM restart.
> AllocatedContainers,ContainersLaunched,AllocatedGB,AvailableGB,AllocatedVCores,AvailableVCores
> in metrics also need to recover when NM restart.
> This should be done in ContainerManagerImpl#recoverContainer.
> The scenario could be reproduction by the following steps:
> # Make sure
> YarnConfiguration.NM_RECOVERY_ENABLED=true,YarnConfiguration.NM_RECOVERY_SUPERVISED=true
> in NM
> # Submit an application and keep running
> # Restart NM
> # Stop the application
> # Now you get the negative values
> {code}
> /jmx?qry=Hadoop:service=NodeManager,name=NodeManagerMetrics
> {code}
> {code}
> {
> name: "Hadoop:service=NodeManager,name=NodeManagerMetrics",
> modelerType: "NodeManagerMetrics",
> tag.Context: "yarn",
> tag.Hostname: "hadoop.com",
> ContainersLaunched: 0,
> ContainersCompleted: 0,
> ContainersFailed: 2,
> ContainersKilled: 0,
> ContainersIniting: 0,
> ContainersRunning: 0,
> AllocatedGB: 0,
> AllocatedContainers: -2,
> AvailableGB: 160,
> AllocatedVCores: -11,
> AvailableVCores: 3611,
> ContainerLaunchDurationNumOps: 2,
> ContainerLaunchDurationAvgTime: 6,
> BadLocalDirs: 0,
> BadLogDirs: 0,
> GoodLocalDirsDiskUtilizationPerc: 2,
> GoodLogDirsDiskUtilizationPerc: 2
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8592) [UI2] rmip:port/ui2 endpoint shows a blank page in windows OS and Chrome browser
[
https://issues.apache.org/jira/browse/YARN-8592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566629#comment-16566629
]
Hudson commented on YARN-8592:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14694 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/14694/])
YARN-8592. [UI2] rmip:port/ui2 endpoint shows a blank page in windows OS
(sunilg: rev 97870ec1f6e40ee86f29411d71b6c687bbed)
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/utils/date-utils.js
> [UI2] rmip:port/ui2 endpoint shows a blank page in windows OS and Chrome
> browser
>
>
> Key: YARN-8592
> URL: https://issues.apache.org/jira/browse/YARN-8592
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Akhil S Naik
>Assignee: Akhil PB
>Priority: Major
> Fix For: 3.2.0, 3.1.2
>
> Attachments: YARN-8592.001.patch
>
>
> after successfully installing ambari 2.7v and hdp 3.0 in Windows Operating
> Sytem .
> But, When we connect to Resource Manager UI(v2), web browser shows a blank
> page.
> In a chrome development tool, this error is shown
> Stack trace..
> {code:java}
> TypeError: Cannot read property 'join's null
> at new i (hostname:8088/ui2/assets/vendor.js:5598:40988)
> at l (hostname:8088/ui2/assets/vendor.js:5598:41338)
> at p (hostname:8088/ui2/assets/vendor.js:5598:42035)
> at Function.q [as guess] (hostname:8088/ui2/assets/vendor.js:5598:42235)
> at getDefaultTimezone (hostname:8088/ui2/assets/yarn-ui.js:366:445)
> at convertTimestampWithT (hostname:8088/ui2/assets/yarn-ui.js:366:656)
> at Object.timeStampToDate (hostname:8088/ui2/assets/yarn-ui.js:348:127)
> at Class.internalNormalizeSingleResponse
> (hostname:8088/ui2/assets/yarn-ui.js:287:73)
> at Class. (hostname:8088/ui2/assets/yarn-ui.js:288:2024)
> at Array.map (native)
> {code}
> The Error is hit in the underlying framework that Yarn-UI uses :
> moment-timezone-with-data-2010-2020.min.js
> this is the discussion happening in framework github for the same :
> https://github.com/moment/moment-timezone/issues/294
> Please find the discussion on this bug here :
> https://community.hortonworks.com/questions/208887/resource-manager-web-ui-v2-shows-a-blank-page.html
> It works well in firefox browser in windows Operating systems.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8592) [UI2] rmip:port/ui2 endpoint shows a blank page in windows OS and Chrome browser
[
https://issues.apache.org/jira/browse/YARN-8592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil Govindan updated YARN-8592:
-
Summary: [UI2] rmip:port/ui2 endpoint shows a blank page in windows OS and
Chrome browser (was: [UI2] Resource Manager Web UI V2 shows a blank page in
windows OS And Chrome browser)
> [UI2] rmip:port/ui2 endpoint shows a blank page in windows OS and Chrome
> browser
>
>
> Key: YARN-8592
> URL: https://issues.apache.org/jira/browse/YARN-8592
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Akhil S Naik
>Assignee: Akhil PB
>Priority: Major
> Attachments: YARN-8592.001.patch
>
>
> after successfully installing ambari 2.7v and hdp 3.0 in Windows Operating
> Sytem .
> But, When we connect to Resource Manager UI(v2), web browser shows a blank
> page.
> In a chrome development tool, this error is shown
> Stack trace..
> {code:java}
> TypeError: Cannot read property 'join's null
> at new i (hostname:8088/ui2/assets/vendor.js:5598:40988)
> at l (hostname:8088/ui2/assets/vendor.js:5598:41338)
> at p (hostname:8088/ui2/assets/vendor.js:5598:42035)
> at Function.q [as guess] (hostname:8088/ui2/assets/vendor.js:5598:42235)
> at getDefaultTimezone (hostname:8088/ui2/assets/yarn-ui.js:366:445)
> at convertTimestampWithT (hostname:8088/ui2/assets/yarn-ui.js:366:656)
> at Object.timeStampToDate (hostname:8088/ui2/assets/yarn-ui.js:348:127)
> at Class.internalNormalizeSingleResponse
> (hostname:8088/ui2/assets/yarn-ui.js:287:73)
> at Class. (hostname:8088/ui2/assets/yarn-ui.js:288:2024)
> at Array.map (native)
> {code}
> The Error is hit in the underlying framework that Yarn-UI uses :
> moment-timezone-with-data-2010-2020.min.js
> this is the discussion happening in framework github for the same :
> https://github.com/moment/moment-timezone/issues/294
> Please find the discussion on this bug here :
> https://community.hortonworks.com/questions/208887/resource-manager-web-ui-v2-shows-a-blank-page.html
> It works well in firefox browser in windows Operating systems.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8592) [UI2] Resource Manager Web UI V2 shows a blank page in windows OS And Chrome browser
[
https://issues.apache.org/jira/browse/YARN-8592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566589#comment-16566589
]
Sunil Govindan commented on YARN-8592:
--
Thanks [~akhilpb]. This looks fine to me.
I tested in a chrome browser and looks fine.
> [UI2] Resource Manager Web UI V2 shows a blank page in windows OS And Chrome
> browser
>
>
> Key: YARN-8592
> URL: https://issues.apache.org/jira/browse/YARN-8592
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Akhil S Naik
>Assignee: Akhil PB
>Priority: Major
> Attachments: YARN-8592.001.patch
>
>
> after successfully installing ambari 2.7v and hdp 3.0 in Windows Operating
> Sytem .
> But, When we connect to Resource Manager UI(v2), web browser shows a blank
> page.
> In a chrome development tool, this error is shown
> Stack trace..
> {code:java}
> TypeError: Cannot read property 'join's null
> at new i (hostname:8088/ui2/assets/vendor.js:5598:40988)
> at l (hostname:8088/ui2/assets/vendor.js:5598:41338)
> at p (hostname:8088/ui2/assets/vendor.js:5598:42035)
> at Function.q [as guess] (hostname:8088/ui2/assets/vendor.js:5598:42235)
> at getDefaultTimezone (hostname:8088/ui2/assets/yarn-ui.js:366:445)
> at convertTimestampWithT (hostname:8088/ui2/assets/yarn-ui.js:366:656)
> at Object.timeStampToDate (hostname:8088/ui2/assets/yarn-ui.js:348:127)
> at Class.internalNormalizeSingleResponse
> (hostname:8088/ui2/assets/yarn-ui.js:287:73)
> at Class. (hostname:8088/ui2/assets/yarn-ui.js:288:2024)
> at Array.map (native)
> {code}
> The Error is hit in the underlying framework that Yarn-UI uses :
> moment-timezone-with-data-2010-2020.min.js
> this is the discussion happening in framework github for the same :
> https://github.com/moment/moment-timezone/issues/294
> Please find the discussion on this bug here :
> https://community.hortonworks.com/questions/208887/resource-manager-web-ui-v2-shows-a-blank-page.html
> It works well in firefox browser in windows Operating systems.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8613) Old RM UI shows wrong vcores total value
[ https://issues.apache.org/jira/browse/YARN-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566557#comment-16566557 ] Bibin A Chundatt commented on YARN-8613: [~sunilg] Yes . Looks like queue metrics issue . IIRC the Cluster Resource is shown from QueueMetrics , and reservation used on create issue earlier . > Old RM UI shows wrong vcores total value > > > Key: YARN-8613 > URL: https://issues.apache.org/jira/browse/YARN-8613 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akhil PB >Priority: Major > Attachments: Screen Shot 2018-08-02 at 12.12.41 PM.png, Screen Shot > 2018-08-02 at 12.16.53 PM.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8615) [UI2] Resource Usage tab shows only memory related info. No info available for vcores/gpu.
[
https://issues.apache.org/jira/browse/YARN-8615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566547#comment-16566547
]
genericqa commented on YARN-8615:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
39m 14s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 39s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | YARN-8615 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12934035/YARN-8615.001.patch |
| Optional Tests | asflicense shadedclient |
| uname | Linux b8c85bea504d 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 1ea8116 |
| maven | version: Apache Maven 3.3.9 |
| Max. process+thread count | 303 (vs. ulimit of 1) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/21484/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> [UI2] Resource Usage tab shows only memory related info. No info available
> for vcores/gpu.
> --
>
> Key: YARN-8615
> URL: https://issues.apache.org/jira/browse/YARN-8615
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn-ui-v2
>Reporter: Akhil PB
>Assignee: Akhil PB
>Priority: Major
> Attachments: YARN-8615.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8615) [UI2] Resource Usage tab shows only memory related info. No info available for vcores/gpu.
[ https://issues.apache.org/jira/browse/YARN-8615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akhil PB updated YARN-8615: --- Summary: [UI2] Resource Usage tab shows only memory related info. No info available for vcores/gpu. (was: [UI2] Resource Usage tab shows only memory related info. Drop Down box disappears to select vcores/gpu) > [UI2] Resource Usage tab shows only memory related info. No info available > for vcores/gpu. > -- > > Key: YARN-8615 > URL: https://issues.apache.org/jira/browse/YARN-8615 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-ui-v2 >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8615.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8615) [UI2] Resource Usage tab shows only memory related info. Drop Down box disappears to select vcores/gpu
Akhil PB created YARN-8615: -- Summary: [UI2] Resource Usage tab shows only memory related info. Drop Down box disappears to select vcores/gpu Key: YARN-8615 URL: https://issues.apache.org/jira/browse/YARN-8615 Project: Hadoop YARN Issue Type: Bug Components: yarn-ui-v2 Reporter: Akhil PB Assignee: Akhil PB -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8509) Fix UserLimit calculation for preemption to balance scenario after queue satisfied
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566490#comment-16566490 ] Zian Chen commented on YARN-8509: - Thanks [~csingh] for the review. The failed UT are not related. [~sunilg], can you help commit the patch if everything looks good? Thanks > Fix UserLimit calculation for preemption to balance scenario after queue > satisfied > > > Key: YARN-8509 > URL: https://issues.apache.org/jira/browse/YARN-8509 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Zian Chen >Assignee: Zian Chen >Priority: Major > Attachments: YARN-8509.001.patch, YARN-8509.002.patch, > YARN-8509.003.patch > > > In LeafQueue#getTotalPendingResourcesConsideringUserLimit, we calculate total > pending resource based on user-limit percent and user-limit factor which will > cap pending resource for each user to the minimum of user-limit pending and > actual pending. This will prevent queue from taking more pending resource to > achieve queue balance after all queue satisfied with its ideal allocation. > > We need to change the logic to let queue pending can go beyond userlimit. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8614) Some typos in YarnConfiguration
[
https://issues.apache.org/jira/browse/YARN-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566488#comment-16566488
]
genericqa commented on YARN-8614:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 58s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 54s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
41s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 60m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | YARN-8614 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12934024/YARN-8614.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 5c3da9f462ff 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 1ea8116 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_171 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/21483/testReport/ |
| Max. process+thread count | 301 (vs. ulimit of 1) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/21483/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Some typos in
[jira] [Commented] (YARN-8594) [UI2] Display current logged in user
[ https://issues.apache.org/jira/browse/YARN-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566451#comment-16566451 ] Hudson commented on YARN-8594: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14693 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14693/]) YARN-8594. [UI2] Display current logged in user. Contributed by Akhil (sunilg: rev 1ea81169bad5bd6433348ef8e5e7ac12c5a9cb5e) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/styles/app.scss * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/templates/application.hbs * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/adapters/cluster-user-info.js * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/serializers/cluster-user-info.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/controllers/application.js * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/routes/application.js * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/app/models/cluster-user-info.js > [UI2] Display current logged in user > > > Key: YARN-8594 > URL: https://issues.apache.org/jira/browse/YARN-8594 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Fix For: 3.2.0, 3.1.2 > > Attachments: YARN-8594.001.patch, YARN-8594.002.patch, > YARN-8594.003.patch, YARN-8594.004.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8614) Some typos in YarnConfiguration
[ https://issues.apache.org/jira/browse/YARN-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sen Zhao reassigned YARN-8614: -- Assignee: Sen Zhao > Some typos in YarnConfiguration > --- > > Key: YARN-8614 > URL: https://issues.apache.org/jira/browse/YARN-8614 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sen Zhao >Assignee: Sen Zhao >Priority: Minor > Attachments: YARN-8614.001.patch > > > Fix some typos in comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8614) Some typos in YarnConfiguration
[ https://issues.apache.org/jira/browse/YARN-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sen Zhao updated YARN-8614: --- Attachment: YARN-8614.001.patch > Some typos in YarnConfiguration > --- > > Key: YARN-8614 > URL: https://issues.apache.org/jira/browse/YARN-8614 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sen Zhao >Priority: Minor > Attachments: YARN-8614.001.patch > > > Fix some typos in comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8614) Some typos in YarnConfiguration
Sen Zhao created YARN-8614: -- Summary: Some typos in YarnConfiguration Key: YARN-8614 URL: https://issues.apache.org/jira/browse/YARN-8614 Project: Hadoop YARN Issue Type: Bug Reporter: Sen Zhao Fix some typos in comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8594) [UI2] Display current logged in user
[ https://issues.apache.org/jira/browse/YARN-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566434#comment-16566434 ] Sunil Govindan commented on YARN-8594: -- Looks good +1 > [UI2] Display current logged in user > > > Key: YARN-8594 > URL: https://issues.apache.org/jira/browse/YARN-8594 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8594.001.patch, YARN-8594.002.patch, > YARN-8594.003.patch, YARN-8594.004.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8613) Old RM UI shows wrong vcores total value
[
https://issues.apache.org/jira/browse/YARN-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566433#comment-16566433
]
Sen Zhao commented on YARN-8613:
Is it possible to use {{rs.getClusterResource().getVirtualCores()}} instead of
the corresponding total resource?
> Old RM UI shows wrong vcores total value
>
>
> Key: YARN-8613
> URL: https://issues.apache.org/jira/browse/YARN-8613
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Akhil PB
>Priority: Major
> Attachments: Screen Shot 2018-08-02 at 12.12.41 PM.png, Screen Shot
> 2018-08-02 at 12.16.53 PM.png
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8594) [UI2] Display current logged in user
[ https://issues.apache.org/jira/browse/YARN-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil Govindan updated YARN-8594: - Summary: [UI2] Display current logged in user (was: [UI2] Show the current logged in user in UI2) > [UI2] Display current logged in user > > > Key: YARN-8594 > URL: https://issues.apache.org/jira/browse/YARN-8594 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akhil PB >Assignee: Akhil PB >Priority: Major > Attachments: YARN-8594.001.patch, YARN-8594.002.patch, > YARN-8594.003.patch, YARN-8594.004.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8594) [UI2] Show the current logged in user in UI2
[
https://issues.apache.org/jira/browse/YARN-8594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566429#comment-16566429
]
genericqa commented on YARN-8594:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
39m 25s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 38s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:ba1ab08 |
| JIRA Issue | YARN-8594 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12934015/YARN-8594.004.patch |
| Optional Tests | asflicense shadedclient |
| uname | Linux 3cc0293756c5 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 41da205 |
| maven | version: Apache Maven 3.3.9 |
| Max. process+thread count | 302 (vs. ulimit of 1) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/21482/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> [UI2] Show the current logged in user in UI2
>
>
> Key: YARN-8594
> URL: https://issues.apache.org/jira/browse/YARN-8594
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Akhil PB
>Assignee: Akhil PB
>Priority: Major
> Attachments: YARN-8594.001.patch, YARN-8594.002.patch,
> YARN-8594.003.patch, YARN-8594.004.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8613) Old RM UI shows wrong vcores total value
[ https://issues.apache.org/jira/browse/YARN-8613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16566422#comment-16566422 ] Sunil Govindan commented on YARN-8613: -- This looks like a QueuMetrics issue. cc [~rohithsharma] [~leftnoteasy] [~bibinchundatt] > Old RM UI shows wrong vcores total value > > > Key: YARN-8613 > URL: https://issues.apache.org/jira/browse/YARN-8613 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akhil PB >Priority: Major > Attachments: Screen Shot 2018-08-02 at 12.12.41 PM.png, Screen Shot > 2018-08-02 at 12.16.53 PM.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
