[jira] [Created] (YARN-10540) Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes

[Sunil G (Jira)]

Sunil G created YARN-10540:
--

 Summary: Node page is broken in YARN UI1 and UI2 including 
RMWebService api for nodes
 Key: YARN-10540
 URL: https://issues.apache.org/jira/browse/YARN-10540
 Project: Hadoop YARN
  Issue Type: Task
  Components: webapp
Affects Versions: 3.2.2
Reporter: Sunil G


YARN-10450 added changes in NodeInfo class.

Various exceptions are showing while accessing UI2 and UI1 NODE pages. 
{code:java}
Caused by: java.lang.NullPointerException
at 
org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NodeInfo.(NodeInfo.java:103)
at 
org.apache.hadoop.yarn.server.resourcemanager.webapp.NodesPage$NodesBlock.render(NodesPage.java:164)
at 
org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
at 
org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
at org.apache.hadoop.yarn.webapp.View.render(View.java:243)
at 
org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
at 
org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117)
at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848)
at 
org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:216)
at 
org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.nodes(RmController.java:70)
 {code}
{code:java}
2020-12-19 22:55:54,846 WARN 
org.apache.hadoop.yarn.webapp.GenericExceptionHandler: INTERNAL_SERVER_ERROR
java.lang.NullPointerException
at 
org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NodeInfo.(NodeInfo.java:103)
at 
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.getNodes(RMWebServices.java:450)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-10247) Application priority queue ACLs are not respected

[Sunil G (Jira)]

Sunil G created YARN-10247:
--

 Summary: Application priority queue ACLs are not respected
 Key: YARN-10247
 URL: https://issues.apache.org/jira/browse/YARN-10247
 Project: Hadoop YARN
  Issue Type: Task
  Components: capacity scheduler
Reporter: Sunil G
Assignee: Sunil G


This is a regression from queue path jira.

App priority acls are not working correctly. 
{code:java}
yarn.scheduler.capacity.root.B.acl_application_max_priority=[user=john 
group=users max_priority=4]
{code}
max_priority enforcement is not working. For user john, maximum supported 
priority is 4. However I can submit like priority 6 for this user.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-10154) CS Dynamic Queues cannot be configured with absolute resources

[Sunil G (Jira)]

Sunil G created YARN-10154:
--

 Summary: CS Dynamic Queues cannot be configured with absolute 
resources
 Key: YARN-10154
 URL: https://issues.apache.org/jira/browse/YARN-10154
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 3.1.3
Reporter: Sunil G
Assignee: Sunil G


In CS, ManagedParent Queue and its template cannot take absolute resource value 
like 
[memory=8192,vcores=8]


 Thsi Jira is to track and improve the configuration reading module of 
DynamicQueue to support absolute resource values.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8293) In YARN Services UI, "User Name for service" should be completely removed in secure clusters

[Sunil G (JIRA)]

Sunil G created YARN-8293:
-

 Summary: In YARN Services UI, "User Name for service" should be 
completely removed in secure clusters
 Key: YARN-8293
 URL: https://issues.apache.org/jira/browse/YARN-8293
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn-ui-v2
Reporter: Sunil G
Assignee: Sunil G


"User Name for service" should be completely removed in secure clusters.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8260) [UI2] Per-application tracking URL is no longer available in YARN UI2

[Sunil G (JIRA)]

Sunil G created YARN-8260:
-

 Summary: [UI2] Per-application tracking URL is no longer available 
in YARN UI2
 Key: YARN-8260
 URL: https://issues.apache.org/jira/browse/YARN-8260
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: yarn-ui-v2
Reporter: Sunil G
Assignee: Sunil G
 Attachments: YARN-8260.001.patch

In 'earlier' yarn UI, on selecting a yarn application, tracking URL is 
prominently available in 'Application overview' allowing for easy navigation to 
AM UI.

In ui2, this is no longer exposed .



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8258) [UI2] New UI webappcontext should inherit all filters from default context

[Sunil G (JIRA)]

Sunil G created YARN-8258:
-

 Summary: [UI2] New UI webappcontext should inherit all filters 
from default context
 Key: YARN-8258
 URL: https://issues.apache.org/jira/browse/YARN-8258
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: webapp
Reporter: Sunil G
Assignee: Sunil G


Thanks [~ssath...@hortonworks.com] for finding this.

Ideally all filters from default context has to be inherited to UI2 context as 
well.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8249) Few REST api's in RMWebServices are missing static user check

[Sunil G (JIRA)]

Sunil G created YARN-8249:
-

 Summary: Few REST api's in RMWebServices are missing static user 
check
 Key: YARN-8249
 URL: https://issues.apache.org/jira/browse/YARN-8249
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: webapp, yarn
Affects Versions: 3.1.0
Reporter: Sunil G
Assignee: Sunil G


Few REST endpoints like node label/ scheduler-conf etc are missing static user 
validation check. Writable apis should not be ran for static users. This is not 
a security hole as such as yarn.admin.acls will be configured to certain users 
in secure cluster correctly. However such a check will help to have same 
semantics across various apis. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8239) [UI2] Clicking on Node Manager UI under AM container info / App Attempt page goes to old RM UI

[Sunil G (JIRA)]

Sunil G created YARN-8239:
-

 Summary: [UI2] Clicking on Node Manager UI under AM container info 
/ App Attempt page goes to old RM UI
 Key: YARN-8239
 URL: https://issues.apache.org/jira/browse/YARN-8239
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: yarn-ui-v2
Reporter: Sunil G
Assignee: Sunil G


"Grid View" and in Containers page of both "Grid" and "Graph" views links to 
old ui.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8238) [Umbrella] YARN deep learning framework examples to run on native service

[Sunil G (JIRA)]

Sunil G created YARN-8238:
-

 Summary: [Umbrella] YARN deep learning framework examples to run 
on native service
 Key: YARN-8238
 URL: https://issues.apache.org/jira/browse/YARN-8238
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: yarn-native-services
Reporter: Sunil G
Assignee: Sunil G


Umbrella jira to track various deep learning frameworks which can run on yarn 
native services.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8237) mxnet yarn spec file to add to native service examples

[Sunil G (JIRA)]

Sunil G created YARN-8237:
-

 Summary: mxnet yarn spec file to add to native service examples
 Key: YARN-8237
 URL: https://issues.apache.org/jira/browse/YARN-8237
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn-native-services
Reporter: Sunil G
Assignee: Sunil G


Mxnet could be run on YARN. This jira will help to add examples, yarnfile, 
docker files which are needed to run Mxnet on YARN.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8236) Invalid kerberos principal file name cause NPE in native service

[Sunil G (JIRA)]

Sunil G created YARN-8236:
-

 Summary: Invalid kerberos principal file name cause NPE in native 
service
 Key: YARN-8236
 URL: https://issues.apache.org/jira/browse/YARN-8236
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn-native-services
Affects Versions: 3.1.0
Reporter: Sunil G


Stack trace

 
{code:java}
2018-04-29 16:22:54,266 WARN webapp.GenericExceptionHandler 
(GenericExceptionHandler.java:toResponse(98)) - INTERNAL_SERVER_ERROR
java.lang.NullPointerException
at 
org.apache.hadoop.yarn.service.client.ServiceClient.addKeytabResourceIfSecure(ServiceClient.java:994)
at 
org.apache.hadoop.yarn.service.client.ServiceClient.submitApp(ServiceClient.java:685)
at 
org.apache.hadoop.yarn.service.client.ServiceClient.actionCreate(ServiceClient.java:269){code}

cc [~gsaha] [~csingh]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8221) RMWebServices also need to honor yarn.resourcemanager.display.per-user-apps

[Sunil G (JIRA)]

Sunil G created YARN-8221:
-

 Summary: RMWebServices also need to honor 
yarn.resourcemanager.display.per-user-apps 
 Key: YARN-8221
 URL: https://issues.apache.org/jira/browse/YARN-8221
 Project: Hadoop YARN
  Issue Type: Bug
  Components: webapp
Reporter: Sunil G
Assignee: Sunil G


yarn.resourcemanager.display.per-user-apps is honored in 
ClientRMService#getApplictions. RMWebServices also need to honor same in 
getApps.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8220) Tensorflow yarn spec file to add to native service examples

[Sunil G (JIRA)]

Sunil G created YARN-8220:
-

 Summary: Tensorflow yarn spec file to add to native service 
examples
 Key: YARN-8220
 URL: https://issues.apache.org/jira/browse/YARN-8220
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn-native-services
Reporter: Sunil G
Assignee: Sunil G


Tensorflow could be run on YARN and could leverage YARN's distributed features.

This spec fill will help to run tf 1.3/1.4 on yarn with cuda8.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-7956) [UI2] Avoid duplicating Components link under Services//Components

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-7956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-7956.
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 3.1.1
   3.2.0

> [UI2] Avoid duplicating Components link under 
> Services//Components
> ---
>
> Key: YARN-7956
> URL: https://issues.apache.org/jira/browse/YARN-7956
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: yarn-ui-v2
>Affects Versions: 3.1.0
>Reporter: Yesha Vora
>Assignee: Yesha Vora
>Priority: Major
> Fix For: 3.2.0, 3.1.1
>
> Attachments: Screen Shot 2018-03-20 at 4.19.37 PM.png, 
> YARN-7956.001.patch
>
>
> Scenario:
> 1) Start Yarn service
> 2) Click on a Running yarn service ( example : yesha-sleeper)
> http://:8088/ui2/#/yarn-app/application_1518804855867_0002/components?service=yesha-sleeper
> 3) Now click on yesha-sleeper [application_1518804855867_0002] link
> Both components and yesha-sleeper [application_1518804855867_0002] link point 
> to one page. 
> HOME/Services/ and HOME/Services//Components refer 
> to same page.
> We should not need two links to refer to one page
> h2.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8189) [UI2] Nodes page column headers are half truncated

[Sunil G (JIRA)]

Sunil G created YARN-8189:
-

 Summary: [UI2] Nodes page column headers are half truncated
 Key: YARN-8189
 URL: https://issues.apache.org/jira/browse/YARN-8189
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 3.1.0
Reporter: Sunil G
Assignee: Sunil G


Increase column width in Node page.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-8125) YARNUIV2 does not failover from standby to active endpoint like previous YARN UI

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-8125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-8125.
---
Resolution: Not A Problem

Pls reopen if there is still issue.

> YARNUIV2 does not failover from standby to active endpoint like previous YARN 
> UI
> 
>
> Key: YARN-8125
> URL: https://issues.apache.org/jira/browse/YARN-8125
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: yarn-ui-v2
>Reporter: Phil Zampino
>Assignee: Sunil G
>Priority: Major
>
> If the YARN UI is accessed via the standby resource manager endpoint, it 
> automatically redirects the requests to the active resource manager endpoint. 
> YARNUIV2 should behave the same way.
> Apache Knox 1.0.0 introduced the ability to dynamically determine proxied 
> RESOURCEMANAGER and YARNUI service endpoints based on YARN configuration from 
> Ambari. This functionality works for RM and YARNUI because even though the 
> YARN config may reference the standby RM endpoint, requests are automatically 
> redirected to the active endpoint.
> RM and YARNUI requests to the standby endpoint result in a HTTP 307 response. 
> YARNUIV2 should respond similarly.
> If YARNUIV2 behaves differently, then Knox will not be able to support its 
> own dynamic configuration behavior when proxying YARNUIV2.
> KNOX-1212 adds the integration with Knox, but KNOX-1236 is blocked by this 
> issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8171) [UI2] AM Node link from attempt page should not redirect to new tab

[Sunil G (JIRA)]

Sunil G created YARN-8171:
-

 Summary: [UI2] AM Node link from attempt page should not redirect 
to new tab
 Key: YARN-8171
 URL: https://issues.apache.org/jira/browse/YARN-8171
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn-ui-v2
Reporter: Sunil G
Assignee: Sunil G


Avoid node link redirection to old UI



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8119) [UI2] Timeline Server address' url scheme should be removed while accessing via KNOX

[Sunil G (JIRA)]

Sunil G created YARN-8119:
-

 Summary: [UI2] Timeline Server address' url scheme should be 
removed while accessing via KNOX
 Key: YARN-8119
 URL: https://issues.apache.org/jira/browse/YARN-8119
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn-ui-v2
Reporter: Sunil G
Assignee: Sunil G


While accessing UI2 behind knox, timeline server address will come with URL 
scheme.

Since UI already takes care of URL scheme, this need to be skipped.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8115) [UI2] URL data like nodeHTTPAddress must be encoded in UI before using for NM/ATS

[Sunil G (JIRA)]

Sunil G created YARN-8115:
-

 Summary: [UI2] URL data like nodeHTTPAddress must be encoded in UI 
before using for NM/ATS
 Key: YARN-8115
 URL: https://issues.apache.org/jira/browse/YARN-8115
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn-ui-v2
Reporter: Sunil G
Assignee: Sreenath Somarajapuram


nodeHTTPAddress is picked from RM ,but it should be encoded before processing 
it from UI



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8068) Upgrading apps from Hadoop 2.7 based clients to 2.8+ cause NPE in app timeline publish

[Sunil G (JIRA)]

Sunil G created YARN-8068:
-

 Summary: Upgrading apps from Hadoop 2.7 based clients to 2.8+ 
cause NPE in app timeline publish
 Key: YARN-8068
 URL: https://issues.apache.org/jira/browse/YARN-8068
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn
Affects Versions: 2.8.3
Reporter: Sunil G
Assignee: Sunil G


[TimelineServiceV1Publisher|eclipse-javadoc:%E2%98%82=hadoop-yarn-server-resourcemanager/src%5C/main%5C/java%3Corg.apache.hadoop.yarn.server.resourcemanager.metrics%7BTimelineServiceV1Publisher.java%E2%98%83TimelineServiceV1Publisher].appCreated
 will cause NPE as we use like below
{code:java}
entityInfo.put(ApplicationMetricsConstants.APPLICATION_PRIORITY_INFO, 
app.getApplicationPriority().getPriority());{code}
We have to handle this case while recovery.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8040) [UI2] yarn new ui web-app does not respect current pathname for REST api

[Sunil G (JIRA)]

Sunil G created YARN-8040:
-

 Summary: [UI2] yarn new ui web-app does not respect current 
pathname for REST api
 Key: YARN-8040
 URL: https://issues.apache.org/jira/browse/YARN-8040
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn-ui-v2
Reporter: Sunil G
Assignee: Sunil G


When ui2 is accessed behind proxy like knox/nginx, trailing path name should 
not be skipped. However trim of "ui2" if its there.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7993) [UI2] yarn-service page need to consider ServiceState to show stop/delete buttons

[Sunil G (JIRA)]

Sunil G created YARN-7993:
-

 Summary: [UI2] yarn-service page need to consider ServiceState to 
show stop/delete buttons
 Key: YARN-7993
 URL: https://issues.apache.org/jira/browse/YARN-7993
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: yarn-ui-v2
Reporter: Sunil G


yarn service page has stop/delete buttons. These buttons has to be shown/hidden 
based on ServiceState of each app from ATS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7912) While launching Native Service app from UI, consider service owner name from user.name query parameter

[Sunil G (JIRA)]

Sunil G created YARN-7912:
-

 Summary: While launching Native Service app from UI, consider 
service owner name from user.name query parameter
 Key: YARN-7912
 URL: https://issues.apache.org/jira/browse/YARN-7912
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn-ui-v2
Reporter: Sunil G


As per comments from [~eyang] in YARN-7827, 

"For supporting knox, it would be good for javascript to detect the url 
entering /ui2 and process [user.name|http://user.name/] property.  If there 
isn't one found, then proceed with ajax call to resource manager to find out 
who is the current user to pass the parameter along the rest api calls."

This Jira will track to handle this. This is now pending feasibility check.

Thanks [~eyang] and [~jianhe]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-7770) Support for setting application priority for Distributed Shell jobs

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-7770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-7770.
---
Resolution: Not A Bug

> Support for setting application priority for Distributed Shell jobs
> ---
>
> Key: YARN-7770
> URL: https://issues.apache.org/jira/browse/YARN-7770
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: applications/distributed-shell
>Reporter: Charan Hebri
>Assignee: Sunil G
>Priority: Major
>
> Currently there isn't a way to submit a Distributed Shell job with an 
> application priority like how it is done via the property
> {noformat}
> mapred.job.priority{noformat}
> for MapReduce jobs. Creating this issue to track support for the same.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-7832) Logs page does not work for Running applications

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-7832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-7832.
---
Resolution: Not A Problem

Thanks [~yeshavora] for confirming, This is working fine with Combine System 
Metric Publisher mode

> Logs page does not work for Running applications
> 
>
> Key: YARN-7832
> URL: https://issues.apache.org/jira/browse/YARN-7832
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: yarn-ui-v2
>Affects Versions: 3.0.0
>Reporter: Yesha Vora
>Assignee: Sunil G
>Priority: Critical
> Attachments: Screen Shot 2018-01-26 at 3.28.40 PM.png, 
> YARN-7832.001.patch
>
>
> Scenario
>  * Run yarn service application
>  * When application is Running, go to log page
>  * Select AttemptId and Container Id
> Logs are not showed on UI. It complains "No log data available!"
>  
> Here 
> [http://xxx:8188/ws/v1/applicationhistory/containers/container_e07_1516919074719_0004_01_01/logs?_=1517009230358]
>  API fails with 500 Internal Server Error.
> {"exception":"WebApplicationException","message":"java.io.IOException: 
> ","javaClassName":"javax.ws.rs.WebApplicationException"}
> {code:java}
> GET 
> http://xxx:8188/ws/v1/applicationhistory/containers/container_e07_1516919074719_0004_01_01/logs?_=1517009230358
>  500 (Internal Server Error)
> (anonymous) @ VM779:1
> send @ vendor.js:572
> ajax @ vendor.js:548
> (anonymous) @ vendor.js:5119
> initializePromise @ vendor.js:2941
> Promise @ vendor.js:3005
> ajax @ vendor.js:5117
> ajax @ yarn-ui.js:1
> superWrapper @ vendor.js:1591
> query @ vendor.js:5112
> ember$data$lib$system$store$finders$$_query @ vendor.js:5177
> query @ vendor.js:5334
> fetchLogFilesForContainerId @ yarn-ui.js:132
> showLogFilesForContainerId @ yarn-ui.js:126
> run @ vendor.js:648
> join @ vendor.js:648
> run.join @ vendor.js:1510
> closureAction @ vendor.js:1865
> trigger @ vendor.js:302
> (anonymous) @ vendor.js:339
> each @ vendor.js:61
> each @ vendor.js:51
> trigger @ vendor.js:339
> d.select @ vendor.js:5598
> (anonymous) @ vendor.js:5598
> d.invoke @ vendor.js:5598
> d.trigger @ vendor.js:5598
> e.trigger @ vendor.js:5598
> (anonymous) @ vendor.js:5598
> d.invoke @ vendor.js:5598
> d.trigger @ vendor.js:5598
> (anonymous) @ vendor.js:5598
> dispatch @ vendor.js:306
> elemData.handle @ vendor.js:281{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7866) [UI2] Kerberizing the UI doesn't give any warning or content when UI is accessed without kinit

[Sunil G (JIRA)]

Sunil G created YARN-7866:
-

 Summary: [UI2] Kerberizing the UI doesn't give any warning or 
content when UI is accessed without kinit
 Key: YARN-7866
 URL: https://issues.apache.org/jira/browse/YARN-7866
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn-ui-v2
Reporter: Sunil G
Assignee: Sunil G


Handle 401 error and show in UI



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-7802) [UI2] Application regex search did not work properly with app name

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-7802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-7802.
---
  Resolution: Fixed
Hadoop Flags: Reviewed

Thanks [~Sreenath]

> [UI2] Application regex search did not work properly with app name
> --
>
> Key: YARN-7802
> URL: https://issues.apache.org/jira/browse/YARN-7802
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: yarn-ui-v2
>Reporter: Yesha Vora
>Assignee: Sreenath Somarajapuram
>Priority: Major
> Attachments: YARN-7802.1.patch, YARN-7802.2.patch
>
>
> Steps:
> 1) Start yarn services with "yesha-hbase-retry-2"
> 2) put regex = yesha-hbase-retry-2
> http://host:8088/ui2/#/yarn-apps/apps?searchText=yesha-hbase-retry-2
> Here, the application does not gets listed. The regex work with 
> "yesha-hbase-retry-" input but does not work with full app name.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7863) Modify placement constraints to support node attributes

[Sunil G (JIRA)]

Sunil G created YARN-7863:
-

 Summary: Modify placement constraints to support node attributes
 Key: YARN-7863
 URL: https://issues.apache.org/jira/browse/YARN-7863
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Sunil G
Assignee: Sunil G


This Jira will track to *Modify existing placement constraints to support node 
attributes.*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7862) YARN native service REST endpoint needs user.name as query param

[Sunil G (JIRA)]

Sunil G created YARN-7862:
-

 Summary: YARN native service REST endpoint needs user.name as 
query param
 Key: YARN-7862
 URL: https://issues.apache.org/jira/browse/YARN-7862
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn-native-services
Reporter: Sunil G


While accessing below yarn rest end point with POST method type,
{code:java}
http://rm_ip:8088/app/v1/services{code}
below error is coming in non-secure cluster.
{noformat}
{
"diagnostics": "Null user"
}{noformat}
When *user.name* is provided as query param with *dr.who* we can see that yarn 
started service with proxy user, not dr.who. 

In non-secure cluster, native service should ideally take the user from remote 
ugi.

in secure cluster, its better to derive user from kerberized shell.

 

cc/  [~jianhe] [~eyang]

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7861) [UI2] Logs page shows duplicated containers with ATS

[Sunil G (JIRA)]

Sunil G created YARN-7861:
-

 Summary: [UI2] Logs page shows duplicated containers with ATS
 Key: YARN-7861
 URL: https://issues.apache.org/jira/browse/YARN-7861
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn-ui-v2
 Environment: There were couple of issues:
 # duplicated container listed from RM and ATS in log container list
 # log page has to be cleared every time same page is accessed
Reporter: Sunil G
Assignee: Sunil G






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7795) Fix jenkins issues of YARN-6592 branch

[Sunil G (JIRA)]

Sunil G created YARN-7795:
-

 Summary: Fix jenkins issues of YARN-6592 branch
 Key: YARN-7795
 URL: https://issues.apache.org/jira/browse/YARN-7795
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Sunil G
Assignee: Sunil G


Refer link . Also fix the javadoc errors as well.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7794) SLSRunner is not loading timeline service jars causing failure

[Sunil G (JIRA)]

Sunil G created YARN-7794:
-

 Summary: SLSRunner is not loading timeline service jars causing 
failure
 Key: YARN-7794
 URL: https://issues.apache.org/jira/browse/YARN-7794
 Project: Hadoop YARN
  Issue Type: Bug
  Components: scheduler-load-simulator
Affects Versions: 3.1.0
Reporter: Sunil G


{code:java}

Caused by: java.lang.ClassNotFoundException: 
org.apache.hadoop.yarn.server.timelineservice.collector.TimelineCollector

        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)

        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

        ... 13 more

Exception in thread "pool-2-thread-390" java.lang.NoClassDefFoundError: 
org/apache/hadoop/yarn/server/timelineservice/collector/TimelineCollector

        at 
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:443)

        at 
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:321)

        at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplication(ClientRMService.java:641){code}
We are getting this error while running SLS. new patch of timelineservice under 
share/hadoop/yarn is not loaded in SLS jvm (verified from slsrunner classpath)

cc/ [~rohithsharma]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7792) Merge work for YARN-6592

[Sunil G (JIRA)]

Sunil G created YARN-7792:
-

 Summary: Merge work for YARN-6592
 Key: YARN-7792
 URL: https://issues.apache.org/jira/browse/YARN-7792
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Sunil G


This Jira is to run aggregated YARN-6592 branch patch against trunk and check 
for any jenkins issues.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7753) [UI2] Application logs has to be pulled from ATS 1.5 instead of ATS2

[Sunil G (JIRA)]

Sunil G created YARN-7753:
-

 Summary: [UI2] Application logs has to be pulled from ATS 1.5 
instead of ATS2
 Key: YARN-7753
 URL: https://issues.apache.org/jira/browse/YARN-7753
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn-ui-v2
Reporter: Sunil G
Assignee: Sunil G


Currently UI tries to pull logs from ATS v2. Instead, it should be pulled from 
ATS v1 as ATS2 doesnt have a log story yet.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7699) queueUsagePercentage is coming as INF for getApp REST api call

[Sunil G (JIRA)]

Sunil G created YARN-7699:
-

 Summary: queueUsagePercentage is coming as INF for getApp REST api 
call
 Key: YARN-7699
 URL: https://issues.apache.org/jira/browse/YARN-7699
 Project: Hadoop YARN
  Issue Type: Bug
  Components: webapp
Affects Versions: 3.0.0, 2.9.0
Reporter: Sunil G
Assignee: Sunil G



REST o/p for getApp call where queueUsagePercentage is INF.
{noformat}
{
"app": {
"id": "application_1514964945154_0001",
"user": "hrt_qa",
"name": "DistributedShell",
"queue": "a1",
"state": "RUNNING",
"finalStatus": "UNDEFINED",
"progress": 90.0,
"trackingUI": "ApplicationMaster",
"trackingUrl": 
"http://ctr-e136-1513029738776-28101-01-04.hwx.site:8088/proxy/application_1514964945154_0001/;,
"diagnostics": "",
"clusterId": 1514964945154,
"applicationType": "YARN",
"applicationTags": "",
"priority": 0,
"startedTime": 1514965015754,
"finishedTime": 0,
"elapsedTime": 418844,
"amContainerLogs": 
"http://ctr-e136-1513029738776-28101-01-07.hwx.site:8042/node/containerlogs/container_e43_1514964945154_0001_01_01/hrt_qa;,
"amHostHttpAddress": 
"ctr-e136-1513029738776-28101-01-07.hwx.site:8042",
"allocatedMB": 600,
"allocatedVCores": 1,
"reservedMB": 0,
"reservedVCores": 0,
"runningContainers": 1,
"memorySeconds": 1227070,
"vcoreSeconds": 2041,
"queueUsagePercentage": INF,
"clusterUsagePercentage": 3.2552083,
"resourceSecondsMap": {
"entry": {
"key": "memory-mb",
"value": "1227070"
},
"entry": {
"key": "vcores",
"value": "2041"
}
},
"preemptedResourceMB": 0,
"preemptedResourceVCores": 0,
"numNonAMContainerPreempted": 0,
"numAMContainerPreempted": 0,
"preemptedMemorySeconds": 0,
"preemptedVcoreSeconds": 0,
"preemptedResourceSecondsMap": {

},
"resourceRequests": [
{
"priority": 0,
"resourceName": "*",
"capability": {
"memory": 600,
"vCores": 1
},
"numContainers": 1,
"relaxLocality": true,
"nodeLabelExpression": "x",
"executionTypeRequest": {
"executionType": "GUARANTEED",
"enforceExecutionType": false
},
"enforceExecutionType": false
}
],
"logAggregationStatus": "NOT_START",
"unmanagedApplication": false,
"appNodeLabelExpression": "x",
"amNodeLabelExpression": "x",
"resourceInfo": {
"resourceUsagesByPartition": [
{
"partitionName": "",
"used": {
"memory": 0,
"vCores": 0
},
"reserved": {
"memory": 0,
"vCores": 0
},
"pending": {
"memory": 0,
"vCores": 0
},
"amUsed": {
"memory": 0,
"vCores": 0
},
"amLimit": {
"memory": 0,
"vCores": 0
}
},
{
"partitionName": "x",
"used": {
"memory": 0,
"vCores": 0
},
"reserved": {
"memory": 0,
"vCores": 0
},
"pending": {
"memory": 600,
"vCores": 1
},
"amUsed": {
"memory": 0,
"vCores": 0
},
"amLimit": {
"memory": 0,
"vCores": 0
}
},
{
"partitionName": "y",
"used": {
"memory": 600,
"vCores": 1
},
"reserved": {
"memory": 0,
"vCores": 0
},
"pending": {
"memory": 0,
"vCores": 0
},
"amUsed": {

[jira] [Resolved] (YARN-7684) The Total Memory and VCores display in the yarn UI is not correct with labeled node

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-7684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-7684.
---
Resolution: Won't Fix

This is already covered in 2.8+ version. YARN-4484 could be backported to 2.7 
as needed.

> The Total Memory  and VCores display in the yarn UI is not correct with 
> labeled node
> 
>
> Key: YARN-7684
> URL: https://issues.apache.org/jira/browse/YARN-7684
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: webapp
>Affects Versions: 2.7.3
>Reporter: Zhao Yi Ming
>Assignee: Zhao Yi Ming
> Fix For: 2.8.0
>
> Attachments: TestResultsAfterPortingYARN-4484.pdf, 
> YARN-7684-branch-2.7.3.001.patch, yarn_issue.pdf
>
>
> The Total Memory  and VCores display in the yarn UI is not correct with 
> labeled node
> recreate steps:
> 1. should have a hadoop cluster
> 2. enabled the yarn Node Labels feature
> 3. create a label eg: yarn rmadmin -addToClusterNodeLabels "test"
> 4. add a node into the label eg: yarn rmadmin -replaceLabelsOnNode 
> "zhaoyim02.com=test"
> 5. then go to the yarn UI http://:8088/cluster/nodes



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7666) Introduce scheduler specific environment variable support in ASC for better scheduling placement configurations

[Sunil G (JIRA)]

Sunil G created YARN-7666:
-

 Summary: Introduce scheduler specific environment variable support 
in ASC for better scheduling placement configurations
 Key: YARN-7666
 URL: https://issues.apache.org/jira/browse/YARN-7666
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Sunil G
Assignee: Sunil G


Introduce a scheduler specific key-value map to hold env variables in ASC.

And also convert AppPlacementAllocator initialization to each app based on 
policy configured at each app.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7651) branch-2 application master (MR) cannot run in 3.1 cluster

[Sunil G (JIRA)]

Sunil G created YARN-7651:
-

 Summary: branch-2 application master (MR) cannot run in 3.1 cluster
 Key: YARN-7651
 URL: https://issues.apache.org/jira/browse/YARN-7651
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager
Affects Versions: 3.1.0
Reporter: Sunil G
Priority: Blocker


{noformat}
2017-12-13 19:21:20,452 WARN [main] org.apache.hadoop.util.NativeCodeLoader: 
Unable to load native-hadoop library for your platform... using builtin-java 
classes where applicable
2017-12-13 19:21:20,481 FATAL [main] 
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.lang.RuntimeException: Unable to determine current user
at 
org.apache.hadoop.conf.Configuration$Resource.getRestrictParserDefault(Configuration.java:253)
at 
org.apache.hadoop.conf.Configuration$Resource.(Configuration.java:219)
at 
org.apache.hadoop.conf.Configuration$Resource.(Configuration.java:211)
at 
org.apache.hadoop.conf.Configuration.addResource(Configuration.java:876)
at 
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1571)
Caused by: java.io.IOException: Exception reading 
/Users/sunilgovindan/install/hadoop/tmp/nm-local-dir/usercache/sunilgovindan/appcache/application_1513172966925_0001/container_1513172966925_0001_01_01/container_tokens
at 
org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:208)
at 
org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:870)
at 
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:803)
at 
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:676)
at 
org.apache.hadoop.conf.Configuration$Resource.getRestrictParserDefault(Configuration.java:251)
... 4 more
Caused by: java.io.IOException: Unknown version 1 in token storage.
at 
org.apache.hadoop.security.Credentials.readTokenStorageStream(Credentials.java:226)
at 
org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:205)
... 8 more
2017-12-13 19:21:20,484 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting 
with status 1
{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7650) Queue link in old RM UI is not working correctly when node labels are enabled

[Sunil G (JIRA)]

Sunil G created YARN-7650:
-

 Summary: Queue link in old RM UI is not working correctly when 
node labels are enabled
 Key: YARN-7650
 URL: https://issues.apache.org/jira/browse/YARN-7650
 Project: Hadoop YARN
  Issue Type: Bug
  Components: webapp
Affects Versions: 2.8.0
Reporter: Sunil G


When Queue link is clicked from application page, 
{{http://hw12715:8088/cluster/app/application_1513168152296_0001}}, we are 
getting below URL

{code}
http://hw12715:8088/cluster/scheduler?openQueues=a1
{code}

It should be like below when node labels are enabled else Queue wont be opened 
up.
{code}
http://localhost:8088/cluster/scheduler?openQueues=a1#Partition: x #Queue: 
root#Queue: a#Queue: a.a1
{code}






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7640) Retrospect to enable CORS related configs by default in YARN

[Sunil G (JIRA)]

Sunil G created YARN-7640:
-

 Summary: Retrospect to enable CORS related configs by default in 
YARN
 Key: YARN-7640
 URL: https://issues.apache.org/jira/browse/YARN-7640
 Project: Hadoop YARN
  Issue Type: Bug
  Components: nodemanager, resourcemanager
Affects Versions: 3.0.0-beta1
Reporter: Sunil G


Currently admin has to do below config changes to enable CORS in YARN.

{code}
1. Add org.apache.hadoop.security.HttpCrossOriginFilterInitializer to 
hadoop.http.filter.initializers
2. Set hadoop.http.cross-origin.enabled to true
3. Set hadoop.http.cross-origin.allowed-methods to GET,HEAD
4. Set yarn.nodemanager.webapp.cross-origin.enabled to true 
5. Set yarn.resourcemanager.webapp.cross-origin.enabled to true
{code}

For better handling, we could enable this config by default in YARN.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7637) GPU volume creation command fails when work preserving is disabled at NM

[Sunil G (JIRA)]

Sunil G created YARN-7637:
-

 Summary: GPU volume creation command fails when work preserving is 
disabled at NM
 Key: YARN-7637
 URL: https://issues.apache.org/jira/browse/YARN-7637
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: nodemanager
Affects Versions: 3.1.0
Reporter: Sunil G
Assignee: Zian Chen


When work preserving is disabled, NM uses {{NMNullStateStoreService}}. Hence 
resource mappings related to GPU wont be saved at Container.

This has to  be rechecked and store accordingly.

cc/ [~leftnoteasy] and [~Zian Chen]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7635) TestRMWebServicesSchedulerActivities fails in trunk

[Sunil G (JIRA)]

Sunil G created YARN-7635:
-

 Summary: TestRMWebServicesSchedulerActivities fails in trunk
 Key: YARN-7635
 URL: https://issues.apache.org/jira/browse/YARN-7635
 Project: Hadoop YARN
  Issue Type: Bug
  Components: test
Affects Versions: 3.1.0
Reporter: Sunil G


Error trace.
{code}
[ERROR] Tests run: 18, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 41.297 
s <<< FAILURE! - in 
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivities
[ERROR] 
testClusterScheduler(org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivities)
  Time elapsed: 0.435 s  <<< FAILURE!
java.lang.AssertionError: incorrect number of elements expected:<33> but 
was:<34>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at 
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched.verifySubQueue(TestRMWebServicesCapacitySched.java:363)
at 
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched.verifySubQueue(TestRMWebServicesCapacitySched.java:384)
at 
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched.verifyClusterScheduler(TestRMWebServicesCapacitySched.java:340)
at 
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched.testClusterScheduler(TestRMWebServicesCapacitySched.java:182)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:369)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:275)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:239)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:160)
at 
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:373)
at 
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:334)
at 
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:119)
at 
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407)

{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-7092) Render application specific log under application tab in new YARN UI

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-7092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-7092.
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 3.1.0

Thanks [~akhilpb] for working on this patch and thanks [~skmvasu] for helping 
in rebasing and review. Committed to trunk

> Render application specific log under application tab in new YARN UI
> 
>
> Key: YARN-7092
> URL: https://issues.apache.org/jira/browse/YARN-7092
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: yarn-ui-v2
>Reporter: Akhil PB
>Assignee: Akhil PB
> Fix For: 3.1.0
>
> Attachments: YARN-7092.001.patch
>
>
> Feature to view application logs in new yarn-ui.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7573) Gpu Information page could be empty for nodes without GPU

[Sunil G (JIRA)]

Sunil G created YARN-7573:
-

 Summary: Gpu Information page could be empty for nodes without GPU
 Key: YARN-7573
 URL: https://issues.apache.org/jira/browse/YARN-7573
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: webapp, yarn-ui-v2
Reporter: Sunil G
Assignee: Sunil G


In new YARN UI, node page is not accessible if that node doesnt have any GPU.
Also Under node page, when we click on "List of Containers/Applications", Gpu 
Information left nave is disappearing.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7564) Fix checkstyle issues of YARN-5881 branch

[Sunil G (JIRA)]

Sunil G created YARN-7564:
-

 Summary: Fix checkstyle issues of YARN-5881 branch
 Key: YARN-7564
 URL: https://issues.apache.org/jira/browse/YARN-7564
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Sunil G
Assignee: Sunil G
Priority: Minor


fix jenkins issues.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7544) Configuration cleanup associated to support absolute resources in a queuec

[Sunil G (JIRA)]

Sunil G created YARN-7544:
-

 Summary: Configuration cleanup associated to support absolute 
resources in a queuec
 Key: YARN-7544
 URL: https://issues.apache.org/jira/browse/YARN-7544
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: capacity scheduler
Reporter: Sunil G
Assignee: Sunil G


Its better to reuse existing *capacity* and *max-capacity* instead of 
introducing new config items.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7538) Performance improvement related to absolute min-max resources

[Sunil G (JIRA)]

Sunil G created YARN-7538:
-

 Summary: Performance improvement related to absolute min-max 
resources
 Key: YARN-7538
 URL: https://issues.apache.org/jira/browse/YARN-7538
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: capacity scheduler
Reporter: Sunil G
Assignee: Sunil G


performance improvements.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7483) Test cases cleanup post YARN-5881

[Sunil G (JIRA)]

Sunil G created YARN-7483:
-

 Summary: Test cases cleanup post YARN-5881
 Key: YARN-7483
 URL: https://issues.apache.org/jira/browse/YARN-7483
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: test
Affects Versions: YARN-5881
Reporter: Sunil G
Assignee: Sunil G


Few UT cases has to be cleaned or rewritten cleanly to consider both 
absolute/percentage based configuration.

Related test classes
# TestLeafQueue
# TestCapacityScheduler



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-6627) Use deployed webapp folder to launch new YARN UI

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-6627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-6627.
---
   Resolution: Fixed
Fix Version/s: 2.9.0

Thanks [~asuresh] for backporting this to 2.9.0 (ref: 
d15be3c93a0d03d15eb2407d266680343b51bf15).

Closing this ticket by updating fix version.

> Use deployed webapp folder to launch new YARN UI
> 
>
> Key: YARN-6627
> URL: https://issues.apache.org/jira/browse/YARN-6627
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: yarn-ui-v2
>Reporter: Sunil G
>Assignee: Sunil G
> Fix For: 2.9.0, 3.0.0-alpha4
>
> Attachments: YARN-6627.0001.patch, YARN-6627.0002.patch, 
> YARN-6627.0003.patch, YARN-6627.branch-2.001.patch
>
>
> Currently new ui war file is placed in share/hadoop/yarn folder. Along with 
> this, its better to have ui2 folder placed under share/hadoop/yarn/webapps so 
> that UI could be launched from a defined folder.
> This will also help to make some cutom config related to ui as well.
> cc/[~jianhe]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-1963) Support priorities across applications within the same queue

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-1963.
---
Resolution: Fixed

Closing this issue as all sub-items are done.

> Support priorities across applications within the same queue 
> -
>
> Key: YARN-1963
> URL: https://issues.apache.org/jira/browse/YARN-1963
> Project: Hadoop YARN
>  Issue Type: New Feature
>  Components: api, resourcemanager
>Reporter: Arun C Murthy
>Assignee: Sunil G
> Attachments: 0001-YARN-1963-prototype.patch, YARN Application 
> Priorities Design _02.pdf, YARN Application Priorities Design.pdf, YARN 
> Application Priorities Design_01.pdf
>
>
> It will be very useful to support priorities among applications within the 
> same queue, particularly in production scenarios. It allows for finer-grained 
> controls without having to force admins to create a multitude of queues, plus 
> allows existing applications to continue using existing queues which are 
> usually part of institutional memory.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7411) Inter-Queue preemption's computeFixpointAllocation need to handle absolute resources while computing normalizedGuarantee

[Sunil G (JIRA)]

Sunil G created YARN-7411:
-

 Summary: Inter-Queue preemption's computeFixpointAllocation need 
to handle absolute resources while computing normalizedGuarantee
 Key: YARN-7411
 URL: https://issues.apache.org/jira/browse/YARN-7411
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: resourcemanager
Affects Versions: YARN-5881
Reporter: Sunil G
Assignee: Sunil G


{{normalizedGuarantee}} is computed based on queue's capacity. This has to be 
updated correctly when CS starts to accept queue's capacity in terms of 
absolute resource.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7410) Cleanup FixedValueResource to avoid dependency to ResourceUtils

[Sunil G (JIRA)]

Sunil G created YARN-7410:
-

 Summary: Cleanup FixedValueResource to avoid dependency to 
ResourceUtils
 Key: YARN-7410
 URL: https://issues.apache.org/jira/browse/YARN-7410
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager
Affects Versions: 3.1.0
Reporter: Sunil G
Assignee: Wangda Tan


Currently FixedValue Resource constants has some dependencies to ResourceUtils. 
This jira will help to cleanup this dependencies.
Thanks [~leftnoteasy] for finding the same.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7364) Queue dash board in new YARN UI has incorrect values

[Sunil G (JIRA)]

Sunil G created YARN-7364:
-

 Summary: Queue dash board in new YARN UI has incorrect values
 Key: YARN-7364
 URL: https://issues.apache.org/jira/browse/YARN-7364
 Project: Hadoop YARN
  Issue Type: Bug
  Components: webapp
Reporter: Sunil G
Assignee: Sunil G


Queue dashboard in cluster over page does not show queue metrics.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7332) Compute effectiveCapacity per each resource vector

[Sunil G (JIRA)]

Sunil G created YARN-7332:
-

 Summary: Compute effectiveCapacity per each resource vector
 Key: YARN-7332
 URL: https://issues.apache.org/jira/browse/YARN-7332
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: capacity scheduler
Affects Versions: YARN-5881
Reporter: Sunil G
Assignee: Sunil G


Currently effective capacity uses a generalized approach based on dominance. 
Hence some vectors may not be calculated correctly. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7331) Change in few metrics in new YARN UI related to native-services

[Sunil G (JIRA)]

Sunil G created YARN-7331:
-

 Summary: Change in few metrics in new YARN UI related to 
native-services
 Key: YARN-7331
 URL: https://issues.apache.org/jira/browse/YARN-7331
 Project: Hadoop YARN
  Issue Type: Sub-task
Affects Versions: yarn-native-services
Reporter: Sunil G
Assignee: Sunil G


Few metrics changes.

1. The below metrics need not to show up in UI. These can be removed.
containersRequested
pendingAAContainers
surplusContainers
Containers Failed Since Last Threshold
Pending Anti-Affinity Containers 

2. completedContainers is Renamed to containersSucceeded
3. containersReady is newly added. 
4. “Created Time” in component to be removed. This is not real created time for 
component. 




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-7324) TestCapacitySchedulerWithMultiResourceTypes.testBasicCapacitySchedulerWithProfile fails in trunk

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-7324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-7324.
---
Resolution: Invalid

Marking as invalid as its a branch specific issue.

> TestCapacitySchedulerWithMultiResourceTypes.testBasicCapacitySchedulerWithProfile
>  fails in trunk
> 
>
> Key: YARN-7324
> URL: https://issues.apache.org/jira/browse/YARN-7324
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: capacity scheduler
>Reporter: Sunil G
>Assignee: Sunil G
>
> *Error Message*
> Attempt state is not correct (timeout). expected: but 
> was:
> *Stacktrace*
> {noformat}
> java.lang.AssertionError: Attempt state is not correct (timeout). 
> expected: but was:
>   at 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWithMultiResourceTypes.testBasicCapacitySchedulerWithProfile(TestCapacitySchedulerWithMultiResourceTypes.java:108)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7324) TestCapacitySchedulerWithMultiResourceTypes.testBasicCapacitySchedulerWithProfile fails in trunk

[Sunil G (JIRA)]

Sunil G created YARN-7324:
-

 Summary: 
TestCapacitySchedulerWithMultiResourceTypes.testBasicCapacitySchedulerWithProfile
 fails in trunk
 Key: YARN-7324
 URL: https://issues.apache.org/jira/browse/YARN-7324
 Project: Hadoop YARN
  Issue Type: Bug
  Components: capacity scheduler
Reporter: Sunil G
Assignee: Sunil G


*Error Message*

Attempt state is not correct (timeout). expected: but was:

*Stacktrace*

{noformat}
java.lang.AssertionError: Attempt state is not correct (timeout). 
expected: but was:
at 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWithMultiResourceTypes.testBasicCapacitySchedulerWithProfile(TestCapacitySchedulerWithMultiResourceTypes.java:108)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-7314) Multiple Resource manager test cases failing

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-7314.
---
Resolution: Duplicate

> Multiple Resource manager test cases failing
> 
>
> Key: YARN-7314
> URL: https://issues.apache.org/jira/browse/YARN-7314
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: lovekesh bansal
>Priority: Critical
>
> Following 14 unit tests are failing in the trunk:
>  
> org.apache.hadoop.yarn.server.resourcemanager.TestApplicationMasterService.testResourceTypes
>  
>  
> org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart.testSchedulerRecovery[FAIR]
> 
>  
> org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart.testDynamicQueueRecovery[FAIR]
>  
>  
> org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter.testRMWritingMassiveHistoryForFairSche
>   
>  
> org.apache.hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter.testRMWritingMassiveHistoryForCapacitySche
>   
>  
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppAttempt.testHeadroomWithBlackListedNodes
>
>  
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testQueueMaxAMShare
>
>  
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testMultipleCompletedEvent
> 
>  
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testRefreshQueuesWhenRMHA
>  
>  
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testQueueMaxAMShareWithContainerReservation
>
>  
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testComputeMaxAMResource
>   
>  
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testQueueMaxAMShareDefault
> 
>  
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testReservationMetrics
> 
>  
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testRequestAMResourceInZeroFairShareQueue
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-7287) Fix typo in ContainerLaunch

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-7287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-7287.
---
Resolution: Duplicate

Marking as duplicate to YARN-7279

> Fix typo in ContainerLaunch
> ---
>
> Key: YARN-7287
> URL: https://issues.apache.org/jira/browse/YARN-7287
> Project: Hadoop YARN
>  Issue Type: Bug
>Reporter: Ajay Kumar
>Priority: Trivial
>
> Fix typo in ContainerLaunch.
> {code}
> .append("  mapreduce.reduce.e nv\n")
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7254) UI and metrics changes related to absolute resource configuration

[Sunil G (JIRA)]

Sunil G created YARN-7254:
-

 Summary: UI and metrics changes related to absolute resource 
configuration
 Key: YARN-7254
 URL: https://issues.apache.org/jira/browse/YARN-7254
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: capacity scheduler
Reporter: Sunil G
Assignee: Sunil G


Impact on UI and metrics related to absolute resource configuration on CS



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-7184) Test Jira

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-7184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-7184.
---
Resolution: Not A Problem

Reopened and closed as "Not a problem"

> Test Jira
> -
>
> Key: YARN-7184
> URL: https://issues.apache.org/jira/browse/YARN-7184
> Project: Hadoop YARN
>  Issue Type: Task
> Environment: Test variable is missing in the test class
>Reporter: lovekesh bansal
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7170) Investigate bower dependencies for YARN UI v2

[Sunil G (JIRA)]

Sunil G created YARN-7170:
-

 Summary: Investigate bower dependencies for YARN UI v2
 Key: YARN-7170
 URL: https://issues.apache.org/jira/browse/YARN-7170
 Project: Hadoop YARN
  Issue Type: Bug
  Components: webapp
Reporter: Sunil G
Assignee: Sunil G


[INFO] bower ember#2.2.0   progress Receiving
objects:  50% (38449/75444), 722.46 MiB | 3.30 MiB/s
...
[INFO] bower ember#2.2.0   progress Receiving
objects:  99% (75017/75444), 1.56 GiB | 3.31 MiB/s

Investigate the dependencies and reduce the download size and speed of 
compilation.

cc/ [~Sreenath] and [~akhilpb]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7157) Support displaying per-user's apps in RM UI page and in secure cluster

[Sunil G (JIRA)]

Sunil G created YARN-7157:
-

 Summary: Support displaying per-user's apps in RM UI page and in 
secure cluster
 Key: YARN-7157
 URL: https://issues.apache.org/jira/browse/YARN-7157
 Project: Hadoop YARN
  Issue Type: Bug
  Components: webapp
Reporter: Sunil G
Assignee: Sunil G


A user who is accessing a secure cluster via a secure UI should be able to see 
only his/her own apps.

This feature will not break any compatibility as it will turned off by default



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-6839) [YARN-3368] Be able to view application / container logs on the new YARN UI.

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-6839.
---
Resolution: Duplicate

> [YARN-3368] Be able to view application / container logs on the new YARN UI.
> 
>
> Key: YARN-6839
> URL: https://issues.apache.org/jira/browse/YARN-6839
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Wangda Tan
>
> Currently viewing application/container logs will be redirected to the old 
> UI, we should leverage the new UI capabilities to make a better log-viewing 
> experiences.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7093) Improve log message in ResourceUtils

[Sunil G (JIRA)]

Sunil G created YARN-7093:
-

 Summary: Improve log message in ResourceUtils
 Key: YARN-7093
 URL: https://issues.apache.org/jira/browse/YARN-7093
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Sunil G
Assignee: Sunil G
Priority: Trivial


Improve log message ResourceUtils class.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7069) [Umbrella] YARN Resource Profiles extended enhancements

[Sunil G (JIRA)]

Sunil G created YARN-7069:
-

 Summary: [Umbrella] YARN Resource Profiles extended enhancements
 Key: YARN-7069
 URL: https://issues.apache.org/jira/browse/YARN-7069
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: nodemanager, resourcemanager
Reporter: Sunil G


This ticket to is to track further enhancements to resource profile work 
happening in YARN-3926



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7056) Document Resource Profiles feature

[Sunil G (JIRA)]

Sunil G created YARN-7056:
-

 Summary: Document Resource Profiles feature
 Key: YARN-7056
 URL: https://issues.apache.org/jira/browse/YARN-7056
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Sunil G






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-6782) ResourceUtils.getResourceTypes() reloads the resource configuration every time

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-6782.
---
Resolution: Done

> ResourceUtils.getResourceTypes() reloads the resource configuration every time
> --
>
> Key: YARN-6782
> URL: https://issues.apache.org/jira/browse/YARN-6782
> Project: Hadoop YARN
>  Issue Type: Sub-task
>  Components: resourcemanager
>Affects Versions: YARN-3926
>Reporter: Daniel Templeton
>Priority: Minor
>  Labels: newbie++
>
> Many objects need to collect the list of configured resources upon creation.  
> The only way do that now is by calling {{getResourceTypes()}}, which reloads 
> the config.  The net result is that the config is loaded and reloaded over 
> and over again during startup.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7039) Fix javac and javadoc errors in YARN-3926 branch

[Sunil G (JIRA)]

Sunil G created YARN-7039:
-

 Summary: Fix javac and javadoc errors in YARN-3926 branch
 Key: YARN-7039
 URL: https://issues.apache.org/jira/browse/YARN-7039
 Project: Hadoop YARN
  Issue Type: Sub-task
Affects Versions: YARN-3926
Reporter: Sunil G
Assignee: Sunil G


Fix javac and doc errors.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7021) TestResourceUtils to be moved to yarn-common

[Sunil G (JIRA)]

Sunil G created YARN-7021:
-

 Summary: TestResourceUtils to be moved to yarn-common
 Key: YARN-7021
 URL: https://issues.apache.org/jira/browse/YARN-7021
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Sunil G


ResourceUtils class is now in yarn-api. Its better its test class also to be 
moved there, however these tests using lot of resources and using 
ConfigurationProvider which is available only in yarn-common.  Hence 
investigate and improve test for ResourceUtils class.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7013) merge related work for YARN-3926 branch

[Sunil G (JIRA)]

Sunil G created YARN-7013:
-

 Summary: merge related work for YARN-3926 branch
 Key: YARN-7013
 URL: https://issues.apache.org/jira/browse/YARN-7013
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Sunil G
Assignee: Sunil G


To run jenkins for whole branch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6892) Improve API implementation in Resources and DominantResourceCalculator in align to ResourceInformation

[Sunil G (JIRA)]

Sunil G created YARN-6892:
-

 Summary: Improve API implementation in Resources and 
DominantResourceCalculator in align to ResourceInformation
 Key: YARN-6892
 URL: https://issues.apache.org/jira/browse/YARN-6892
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Sunil G
Assignee: Sunil G


In YARN-3926, apis in Resources and DRC spents significant cpu cycles in most 
of its api. For better performance, its better to improve the apis as resource 
types order is defined in system level (ResourceUtils class ensures this post 
YARN-6788)

This work is preceding to YARN-6788



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6872) Ensure apps could run given NodeLabels are disabled post RM switchover/restart

[Sunil G (JIRA)]

Sunil G created YARN-6872:
-

 Summary: Ensure apps could run given NodeLabels are disabled post 
RM switchover/restart
 Key: YARN-6872
 URL: https://issues.apache.org/jira/browse/YARN-6872
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager
Reporter: Sunil G
Assignee: Sunil G


Post YARN-6031, few apps could be failed during recovery provided they had some 
label requirements for AM and labels were disable post RM restart/switchover. 
As discussed in YARN-6031, its better to run such apps as it may be long 
running apps as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6789) new api to get all supported resources from RM

[Sunil G (JIRA)]

Sunil G created YARN-6789:
-

 Summary: new api to get all supported resources from RM
 Key: YARN-6789
 URL: https://issues.apache.org/jira/browse/YARN-6789
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Sunil G
Assignee: Sunil G


It will be better to provide an api to get all supported resource types from RM.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6788) Improve performance of resource profile branch

[Sunil G (JIRA)]

Sunil G created YARN-6788:
-

 Summary: Improve performance of resource profile branch
 Key: YARN-6788
 URL: https://issues.apache.org/jira/browse/YARN-6788
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Sunil G
Assignee: Sunil G


Currently we could see a 15% performance delta with this branch. 
Few performance improvements to improve the same.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6747) TestFSAppStarvation.testPreemptionEnable fails intermittently

[Sunil G (JIRA)]

Sunil G created YARN-6747:
-

 Summary: TestFSAppStarvation.testPreemptionEnable fails 
intermittently
 Key: YARN-6747
 URL: https://issues.apache.org/jira/browse/YARN-6747
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Sunil G


*Error Message*
Apps re-added even before starvation delay passed expected:<4> but was:<3>

*Stacktrace*
java.lang.AssertionError: Apps re-added even before starvation delay passed 
expected:<4> but was:<3>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation.testPreemptionEnabled(TestFSAppStarvation.java:117)
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-6663) Backport YARN-5333 to branch-2.8, branch-2.7

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-6663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-6663.
---
Resolution: Done

> Backport YARN-5333 to branch-2.8, branch-2.7
> 
>
> Key: YARN-6663
> URL: https://issues.apache.org/jira/browse/YARN-6663
> Project: Hadoop YARN
>  Issue Type: Task
>Reporter: Rohith Sharma K S
>Assignee: Rohith Sharma K S
> Attachments: YARN-5333-branch-2.7.0001.patch, 
> YARN-5333-branch-2.8.0001.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6627) Use deployed webapp folder to launch new YARN UI

[Sunil G (JIRA)]

Sunil G created YARN-6627:
-

 Summary: Use deployed webapp folder to launch new YARN UI
 Key: YARN-6627
 URL: https://issues.apache.org/jira/browse/YARN-6627
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: yarn-ui-v2
Reporter: Sunil G
Assignee: Sunil G


Currently new ui war file is placed in share/hadoop/yarn folder. Along with 
this, its better to have ui2 folder placed under share/hadoop/yarn/webapps so 
that UI could be launched from a defined folder.

This will also help to make some cutom config related to ui as well.

cc/[~jianhe]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6538) Inter Queue preemption is not happening when DRF is configured

[Sunil G (JIRA)]

Sunil G created YARN-6538:
-

 Summary: Inter Queue preemption is not happening when DRF is 
configured
 Key: YARN-6538
 URL: https://issues.apache.org/jira/browse/YARN-6538
 Project: Hadoop YARN
  Issue Type: Bug
  Components: scheduler preemption
Affects Versions: 2.8.0
Reporter: Sunil G
Assignee: Sunil G


Cluster capacity of . Here memory is more and vcores 
are less. If applications have more demand, vcores might be exhausted. 
Inter queue preemption ideally has to be kicked in once vcores is over 
utilized. However preemption is not happening.

Analysis:
In {{AbstractPreemptableResourceCalculator.computeFixpointAllocation}}, 
{code}
// assign all cluster resources until no more demand, or no resources are
// left
while (!orderedByNeed.isEmpty() && Resources.greaterThan(rc, totGuarant,
unassigned, Resources.none())) {
{code}
 will loop even when vcores are 0. Hence we are having more vcores in 
idealAssigned which cause no-preemption cases.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-6502) branch-2 build is broken

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-6502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-6502.
---
Resolution: Duplicate

Thanks. Closing this as dup.

> branch-2 build is broken
> 
>
> Key: YARN-6502
> URL: https://issues.apache.org/jira/browse/YARN-6502
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: resourcemanager
>Reporter: Sunil G
>Priority: Blocker
>
> {code}
> [ERROR] COMPILATION ERROR :
> [INFO] -
> [ERROR] 
> /home/sunilg/hadoop/repo/branch-2/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSSchedulerNode.java:[183,18]
>  cannot find symbol
>   symbol:   method 
> putIfAbsent(org.apache.hadoop.yarn.api.records.ApplicationAttemptId,org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt)
>   location: variable appIdToAppMap of type 
> java.util.Map
> [ERROR] 
> /home/sunilg/hadoop/repo/branch-2/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSSchedulerNode.java:[184,29]
>  cannot find symbol
>   symbol:   method 
> putIfAbsent(org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt,org.apache.hadoop.yarn.api.records.Resource)
>   location: variable resourcesPreemptedForApp of type 
> java.util.Map
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6502) branch-2 build is broken

[Sunil G (JIRA)]

Sunil G created YARN-6502:
-

 Summary: branch-2 build is broken
 Key: YARN-6502
 URL: https://issues.apache.org/jira/browse/YARN-6502
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager
Reporter: Sunil G
Priority: Blocker


{noformat}
[ERROR] COMPILATION ERROR :
[INFO] -
[ERROR] 
/home/sunilg/hadoop/repo/branch-2/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSSchedulerNode.java:[183,18]
 cannot find symbol
  symbol:   method 
putIfAbsent(org.apache.hadoop.yarn.api.records.ApplicationAttemptId,org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt)
  location: variable appIdToAppMap of type 
java.util.Map
[ERROR] 
/home/sunilg/hadoop/repo/branch-2/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSSchedulerNode.java:[184,29]
 cannot find symbol
  symbol:   method 
putIfAbsent(org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt,org.apache.hadoop.yarn.api.records.Resource)
  location: variable resourcesPreemptedForApp of type 
java.util.Map
{noformat}





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6471) Support to add min/max resource configuration for a queue

[Sunil G (JIRA)]

Sunil G created YARN-6471:
-

 Summary: Support to add min/max resource configuration for a queue
 Key: YARN-6471
 URL: https://issues.apache.org/jira/browse/YARN-6471
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: capacity scheduler
Reporter: Sunil G
Assignee: Sunil G


This jira will track the new configurations which are needed to configure min 
resource and max resource of various resource types in a queue.

For eg: 
{noformat}
yarn.scheduler.capacity.root.default.memory.min-resource
yarn.scheduler.capacity.root.default.memory.max-resource
yarn.scheduler.capacity.root.default.vcores.min-resource
yarn.scheduler.capacity.root.default.vcores.max-resource
{noformat}

Uploading a patch soon



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6402) Move 'Long Running Services' to an independent tab at top level for new Yarn UI

[Sunil G (JIRA)]

Sunil G created YARN-6402:
-

 Summary: Move 'Long Running Services' to an independent tab at top 
level for new Yarn UI
 Key: YARN-6402
 URL: https://issues.apache.org/jira/browse/YARN-6402
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: webapp
Reporter: Sunil G
Assignee: Sunil G


Currently 'Long Running Services' is a sub link in 'Applications' page. 
Services could be made top level



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6398) Implement a new native-service UI

[Sunil G (JIRA)]

Sunil G created YARN-6398:
-

 Summary: Implement a new native-service UI
 Key: YARN-6398
 URL: https://issues.apache.org/jira/browse/YARN-6398
 Project: Hadoop YARN
  Issue Type: Sub-task
  Components: yarn-native-services
Reporter: Sunil G
Assignee: Akhil PB


Create a new and advanced native service UI which can co-exist with the new 
Yarn UI.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-5496) Make Node Heatmap Chart categories clickable in new YARN UI

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-5496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-5496.
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 3.0.0-alpha3

Committed to trunk. Thanks [~GergelyNovak] for the contribution and thanks 
[~akhilpb] for additional review.

> Make Node Heatmap Chart categories clickable in new YARN UI
> ---
>
> Key: YARN-5496
> URL: https://issues.apache.org/jira/browse/YARN-5496
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Yesha Vora
>Assignee: Gergely Novák
> Fix For: 3.0.0-alpha3
>
> Attachments: YARN-5496.001.patch, YARN-5496.002.patch, 
> YARN-5496.003.patch, YARN-5496.004.patch
>
>
> Make Node Heatmap Chart categories clickable. 
> This Heatmap chart has few categories like 10% used, 30% used etc.
> This tags should be clickable. If user clicks on 10% used tag, it should show 
> hosts with 10% usage.  This can be a useful feature for clusters having 1000s 
> of nodes.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6286) TestCapacityScheduler.testAMLimitUsage throws UndeclaredThrowableException

[Sunil G (JIRA)]

Sunil G created YARN-6286:
-

 Summary: TestCapacityScheduler.testAMLimitUsage throws 
UndeclaredThrowableException
 Key: YARN-6286
 URL: https://issues.apache.org/jira/browse/YARN-6286
 Project: Hadoop YARN
  Issue Type: Bug
  Components: test
Reporter: Sunil G


{code}
testAMLimitUsage(org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler)
  Time elapsed: 0.124 sec  <<< ERROR!
java.lang.reflect.UndeclaredThrowableException: null
at 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.validateResourceRequest(SchedulerUtils.java:253)
at 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndValidateRequest(SchedulerUtils.java:218)
at 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndValidateRequest(SchedulerUtils.java:189)
at 
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.validateAndCreateResourceRequest(RMAppManager.java:497)
at 
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:384)
at 
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:295)
at 
org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplication(ClientRMService.java:664)
at 
org.apache.hadoop.yarn.server.resourcemanager.MockRM$2.run(MockRM.java:752)
at 
org.apache.hadoop.yarn.server.resourcemanager.MockRM$2.run(MockRM.java:746)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
at 
org.apache.hadoop.yarn.server.resourcemanager.MockRM.submitApp(MockRM.java:765)
at 
org.apache.hadoop.yarn.server.resourcemanager.MockRM.submitApp(MockRM.java:665)
at 
org.apache.hadoop.yarn.server.resourcemanager.MockRM.submitApp(MockRM.java:572)
at 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler.verifyAMLimitForLeafQueue(TestCapacityScheduler.java:3370)
at 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler.testAMLimitUsage(TestCapacityScheduler.java:3232)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6278) -Pyarn-ui build seems broken in trunk

[Sunil G (JIRA)]

Sunil G created YARN-6278:
-

 Summary: -Pyarn-ui build seems broken in trunk
 Key: YARN-6278
 URL: https://issues.apache.org/jira/browse/YARN-6278
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Sunil G
Priority: Critical


Link to the error is 
[here|https://builds.apache.org/job/PreCommit-HDFS-Build/18535/artifact/patchprocess/patch-compile-root.txt]

{code}
qunit-notifications#0.1.1 bower_components/qunit-notifications
[INFO] 
[INFO] --- exec-maven-plugin:1.3.1:exec (ember build) @ hadoop-yarn-ui ---

/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/target/src/main/webapp/node_modules/merge-trees/index.js:33
class MergeTrees {
^
Unexpected reserved word
SyntaxError: Unexpected reserved word
at Module._compile (module.js:439:25)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at Object. 
(/testptch/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/target/src/main/webapp/node_modules/broccoli-merge-trees/index.js:2:18)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-5493) In leaf queue page, list applications should only show applications from that leaf queues

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-5493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-5493.
---
Resolution: Duplicate

Duplicated to YARN-5335. This was fixed while em-table conversion is done.

> In leaf queue page, list applications should only show applications from that 
> leaf queues
> -
>
> Key: YARN-5493
> URL: https://issues.apache.org/jira/browse/YARN-5493
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Yesha Vora
>Assignee: Yesha Vora
>
> Steps to reproduce:
> * Create a 2 queues
> * Go to leaf queue page at http://:/#/yarn-queue-apps/ 
> url
> * click on application list. 
> Here, it list down all the applications. Instead , It should list down only 
> applications from that particular leaf queue. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-4516) [YARN-3368] Use em-table to better render tables

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-4516.
---
Resolution: Won't Fix

YARN-5335 already handled this part. Now all tables in new UI is using em-table.

> [YARN-3368] Use em-table to better render tables
> 
>
> Key: YARN-4516
> URL: https://issues.apache.org/jira/browse/YARN-4516
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Wangda Tan
>Assignee: Li Lu
>
> Currently we're using DataTables, it isn't integrated to Ember.js very well. 
> Instead we can use em-table (see https://github.com/sreenaths/em-table/wiki, 
> which is created for Tez UI). It supports features such as selectable 
> columns, pagination, etc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6240) TestCapacityScheduler.testRefreshQueuesWithQueueDelete fails randomly

[Sunil G (JIRA)]

Sunil G created YARN-6240:
-

 Summary: TestCapacityScheduler.testRefreshQueuesWithQueueDelete 
fails randomly
 Key: YARN-6240
 URL: https://issues.apache.org/jira/browse/YARN-6240
 Project: Hadoop YARN
  Issue Type: Bug
  Components: test
Reporter: Sunil G


*Error Message*

Expected to NOT throw exception when refresh queue tries to delete a queue 
WITHOUT running apps

Link 
[here|https://builds.apache.org/job/PreCommit-YARN-Build/15092/testReport/org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity/TestCapacityScheduler/testRefreshQueuesWithQueueDelete/]

*Stacktrace*

{code}
java.lang.AssertionError: Expected to NOT throw exception when refresh queue 
tries to delete a queue WITHOUT running apps
at org.junit.Assert.fail(Assert.java:88)
at 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler.testRefreshQueuesWithQueueDelete(TestCapacityScheduler.java:3875)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6215) TestFairSchedulerPreemption.testPreemptionBetweenNonSiblingQueues fails in trunk

[Sunil G (JIRA)]

Sunil G created YARN-6215:
-

 Summary: 
TestFairSchedulerPreemption.testPreemptionBetweenNonSiblingQueues fails in trunk
 Key: YARN-6215
 URL: https://issues.apache.org/jira/browse/YARN-6215
 Project: Hadoop YARN
  Issue Type: Bug
  Components: fairscheduler
Reporter: Sunil G


*Error Message*
Incorrect number of containers on the greedy app expected:<4> but was:<8>
Failed test case 
[link|https://builds.apache.org/job/PreCommit-YARN-Build/15038/testReport/org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair/TestFairSchedulerPreemption/testPreemptionBetweenNonSiblingQueues_FairSharePreemptionWithDRF_/]
*Stacktrace*

java.lang.AssertionError: Incorrect number of containers on the greedy app 
expected:<4> but was:<8>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption.verifyPreemption(TestFairSchedulerPreemption.java:282)
at 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption.testPreemptionBetweenNonSiblingQueues(TestFairSchedulerPreemption.java:323)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6201) Add node label as a query param for nodes REST end point

[Sunil G (JIRA)]

Sunil G created YARN-6201:
-

 Summary: Add node label as a query param for nodes REST end point
 Key: YARN-6201
 URL: https://issues.apache.org/jira/browse/YARN-6201
 Project: Hadoop YARN
  Issue Type: Bug
  Components: webapp
Reporter: Sunil G
Assignee: Sunil G


For {{/nodes}} end point, label could be a query params.

This could help to retrieve Node informations for a give label (with resources 
etc).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6129) Relook at FifoIntraQueuePreemptionPlugin#calculateIdealAssignedResourcePerApp parameter list

[Sunil G (JIRA)]

Sunil G created YARN-6129:
-

 Summary: Relook at 
FifoIntraQueuePreemptionPlugin#calculateIdealAssignedResourcePerApp parameter 
list
 Key: YARN-6129
 URL: https://issues.apache.org/jira/browse/YARN-6129
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager, scheduler preemption
Reporter: Sunil G
Assignee: Sunil G
Priority: Minor


Revisit all argument list of 
FifoIntraQueuePreemptionPlugin#calculateIdealAssignedResourcePerApp as per 
[comment|https://issues.apache.org/jira/browse/YARN-5889?focusedCommentId=15843343=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15843343]
 from [~leftnoteasy] in YARN-5889. {{partitionBasedResource}} seems not used, 
but revisit whether its really needed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6082) Webservice connection gets cutoff when it has to send back a large response (webservice)

[Sunil G (JIRA)]

Sunil G created YARN-6082:
-

 Summary: Webservice connection gets cutoff when it has to send 
back a large response (webservice)
 Key: YARN-6082
 URL: https://issues.apache.org/jira/browse/YARN-6082
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Sunil G
Priority: Critical


{noformat}
2017-01-11 07:17:11,475 WARN  ipc.Server (Server.java:run(2202)) - Large 
response size 4476919 for call 
org.apache.hadoop.yarn.api.ApplicationClientProtocolPB.getApplications from 
172.27.0.101:39950 Call#951474 Retry#0
{noformat}

In such cases, json output will get cutoff and client will not get clean 
response.
For eg:
{noformat}
Unexpected token I in JSON at position 851
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-5275) Timeline application page cannot be loaded when no application submitted/running on the cluster after HADOOP-9613

[Sunil G (JIRA)]

 [ 
https://issues.apache.org/jira/browse/YARN-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sunil G resolved YARN-5275.
---
Resolution: Not A Problem

> Timeline application page cannot be loaded when no application 
> submitted/running on the cluster after HADOOP-9613
> -
>
> Key: YARN-5275
> URL: https://issues.apache.org/jira/browse/YARN-5275
> Project: Hadoop YARN
>  Issue Type: Bug
>Affects Versions: 3.0.0-alpha1
>Reporter: Tsuyoshi Ozawa
>Priority: Critical
>
> After HADOOP-9613, Timeline Web UI has a problem reported by [~leftnoteasy] 
> and [~sunilg]
> {quote}
> when no application submitted/running on the cluster, applications page 
> cannot be loaded. 
> {quote}
> We should investigate the reason and fix it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-6001) Improve moveApplicationQueues command line

[Sunil G (JIRA)]

Sunil G created YARN-6001:
-

 Summary: Improve moveApplicationQueues command line
 Key: YARN-6001
 URL: https://issues.apache.org/jira/browse/YARN-6001
 Project: Hadoop YARN
  Issue Type: Bug
  Components: client
Reporter: Sunil G
Assignee: Sunil G


As an example, below command is used to move an application across queue. 
{noformat}./yarn application -movetoqueue  application_1479894790219_0002 
-queue default{noformat}

Inline  with other application's attribute modification features such as 
updateTimeout or priority, movetoqueue cli command lacks unification and more 
complex or error prone.

Suggesting below cli command which can be used.
{noformat}
./yarn application -appId application_1479894790219_0002 -move default
{noformat}

This is inline with other commands such as 
{noformat}
./yarn application -appId  application_1479894790219_0002 -updatePriority 8 
./yarn application -appId  application_1479894790219_0002 -updateLifetime 10
{noformat}

Old command could be still there, but deprecated.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-5984) Refactor move application across queue's CS level implementation

[Sunil G (JIRA)]

Sunil G created YARN-5984:
-

 Summary: Refactor move application across queue's CS level 
implementation
 Key: YARN-5984
 URL: https://issues.apache.org/jira/browse/YARN-5984
 Project: Hadoop YARN
  Issue Type: Bug
  Components: capacity scheduler, resourcemanager
Reporter: Sunil G
Assignee: Sunil G


Currently we use a top level write lock in CS#moveApplication. Also we are 
using few submission time apis in move. This jira will be focussing on coming 
up with a cleaner implementation for moveApplication and could try to share 
code with FS as possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-5932) Retrospect moveToQueue in align with YARN-5611

[Sunil G (JIRA)]

Sunil G created YARN-5932:
-

 Summary: Retrospect moveToQueue in align with YARN-5611
 Key: YARN-5932
 URL: https://issues.apache.org/jira/browse/YARN-5932
 Project: Hadoop YARN
  Issue Type: Bug
  Components: capacity scheduler, resourcemanager
Reporter: Sunil G
Assignee: Sunil G


All dynamic api's of an application's state change could follow a general 
design approach. Currently priority and app timeouts are following this 
approach all corner cases.
*Steps*
- Do a pre-validate check to ensure that changes are fine.
- Update this information to state-store
- Perform real move operation and update in-memory data structures.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-5889) Improve user-limit calculation in capacity scheduler

[Sunil G (JIRA)]

Sunil G created YARN-5889:
-

 Summary: Improve user-limit calculation in capacity scheduler
 Key: YARN-5889
 URL: https://issues.apache.org/jira/browse/YARN-5889
 Project: Hadoop YARN
  Issue Type: Bug
  Components: capacity scheduler
Reporter: Sunil G
Assignee: Sunil G


Currently user-limit is computed during every heartbeat allocation cycle with a 
write lock. To improve performance, this tickets is focussing on moving 
user-limit calculation out of heartbeat allocation flow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-5869) Introduce Shuffle Speed metrics to identify slow nodes in cluster

[Sunil G (JIRA)]

Sunil G created YARN-5869:
-

 Summary: Introduce Shuffle Speed metrics to identify slow nodes in 
cluster
 Key: YARN-5869
 URL: https://issues.apache.org/jira/browse/YARN-5869
 Project: Hadoop YARN
  Issue Type: Improvement
Reporter: Sunil G
Assignee: Sunil G


This jira proposes new shuffle speed metrics which could help to identify slow 
nodes in the cluster. Existing Shuffle Handler in NM could help to get this 
metrics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org