[jira] [Commented] (YARN-6467) CSQueueMetrics needs to update the current metrics for default partition only
[
https://issues.apache.org/jira/browse/YARN-6467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036588#comment-16036588
]
Hadoop QA commented on YARN-6467:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 5 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 32s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 2 new + 584 unchanged - 6 fixed = 586 total (was 590)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 60 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 39m
40s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 62m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6467 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12871184/YARN-6467.003.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 098e41fe5da7 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6
15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 46f7e91 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/16106/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/16106/artifact/patchprocess/whitespace-eol.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/16106/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/16106/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> CSQueueMetrics needs to update the current metrics for default partition only
> -
>
> Key: YARN-6467
>
[jira] [Commented] (YARN-5647) [Security] Collector side changes for loading auth filters and principals
[ https://issues.apache.org/jira/browse/YARN-5647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036577#comment-16036577 ] Rohith Sharma K S commented on YARN-5647: - Adding to Jian review, # May be you can use YarnConfiguration.timelineServiceV2Enabled(conf) which check both enabled and version_check before throwing exception. # Btw, why do we want to modify TimelineDelegationTokenIdentifier? Who will be calling renew/cancel for timeline token? NodeTimelineCollectorManager.java line no, 101 and 102, # first start webapp and start parent threads? just need to flip it which retain old code. > [Security] Collector side changes for loading auth filters and principals > - > > Key: YARN-5647 > URL: https://issues.apache.org/jira/browse/YARN-5647 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Varun Saxena >Assignee: Varun Saxena > Labels: yarn-5355-merge-blocker > Attachments: YARN-5647-YARN-5355.004.patch, > YARN-5647-YARN-5355.005.patch, YARN-5647-YARN-5355.006.patch, > YARN-5647-YARN-5355.007.patch, YARN-5647-YARN-5355.wip.002.patch, > YARN-5647-YARN-5355.wip.003.patch, YARN-5647-YARN-5355.wip.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6688) Add client interface to know default queue for user
[
https://issues.apache.org/jira/browse/YARN-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036546#comment-16036546
]
Bibin A Chundatt commented on YARN-6688:
[~varun_saxena]
Spark wants to know the default queue name before the application is getting
submitted for user. Currently the default queue in capacity is done using user
group mapping .
Planning to add {{UserInfo}} proto for the same So that later userInfo could
be extended to get the max application per user etc.
> Add client interface to know default queue for user
> ---
>
> Key: YARN-6688
> URL: https://issues.apache.org/jira/browse/YARN-6688
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>
> Currently user gets to know queue placement only once application is
> accepted. Provide an option for client/user to know default queue for user.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6467) CSQueueMetrics needs to update the current metrics for default partition only
[ https://issues.apache.org/jira/browse/YARN-6467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036534#comment-16036534 ] Manikandan R commented on YARN-6467: Thanks [~Naganarasimha] for review. Attaching .003.patch for changes. 1. Taken care. 2. Renamed in both QueueMetrics and CSQueueMetrics. 3. Currently it not being used. Hence removed the method. It has been added as part of YARN-4538 4. Metrics like allocatedContainers and aggregateContainersAllocated are being manipulated only based on partition check - meaning only for "default" partition. For that matter, even for other metrics like pendingMB, pendingVCores are also being modified in the same way. Do we need to capture these info in this JIRA for our future reference? 5. Taken care as part on point no.1 . Aren't we referring the same? 6,7,8: Taken care. Also verified that Checkstyle issues and Junit failures are not related to this patch. > CSQueueMetrics needs to update the current metrics for default partition only > - > > Key: YARN-6467 > URL: https://issues.apache.org/jira/browse/YARN-6467 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 2.8.0, 2.7.3, 3.0.0-alpha2 >Reporter: Naganarasimha G R >Assignee: Manikandan R > Attachments: YARN-6467.001.patch, YARN-6467.001.patch, > YARN-6467.002.patch, YARN-6467.003.patch > > > As a followup to YARN-6195, we need to update existing metrics to only > default Partition. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6467) CSQueueMetrics needs to update the current metrics for default partition only
[ https://issues.apache.org/jira/browse/YARN-6467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikandan R updated YARN-6467: --- Attachment: YARN-6467.003.patch > CSQueueMetrics needs to update the current metrics for default partition only > - > > Key: YARN-6467 > URL: https://issues.apache.org/jira/browse/YARN-6467 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 2.8.0, 2.7.3, 3.0.0-alpha2 >Reporter: Naganarasimha G R >Assignee: Manikandan R > Attachments: YARN-6467.001.patch, YARN-6467.001.patch, > YARN-6467.002.patch, YARN-6467.003.patch > > > As a followup to YARN-6195, we need to update existing metrics to only > default Partition. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6280) Add a query parameter in ResourceManager Cluster Applications REST API to control whether or not returns ResourceRequest
[ https://issues.apache.org/jira/browse/YARN-6280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036532#comment-16036532 ] Rohith Sharma K S commented on YARN-6280: - One quick comment, line no 94, # Keep RESOURCE_REQUESTS as-is. Do not pass any arguments to it. # Given 1st is accepted, remove corresponding codes for constructors. line no 67, nit # need not to have else block. Can you keep 009 patch code and only modify DeSelectType to upper case such as RESOURCE_REQUESTS. > Add a query parameter in ResourceManager Cluster Applications REST API to > control whether or not returns ResourceRequest > > > Key: YARN-6280 > URL: https://issues.apache.org/jira/browse/YARN-6280 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager, restapi >Affects Versions: 2.7.3 >Reporter: Lantao Jin >Assignee: Lantao Jin > Attachments: YARN-6280.001.patch, YARN-6280.002.patch, > YARN-6280.003.patch, YARN-6280.004.patch, YARN-6280.005.patch, > YARN-6280.006.patch, YARN-6280.007.patch, YARN-6280.008.patch, > YARN-6280.009.patch, YARN-6280.010.patch > > > Begin from v2.7, the ResourceManager Cluster Applications REST API returns > ResourceRequest list. It's a very large construction in AppInfo. > As a test, we use below URI to query only 2 results: > http:// address:port>/ws/v1/cluster/apps?states=running,accepted&limit=2 > The results are very different: > ||Hadoop version|Total Character|Total Word|Total Lines|Size|| > |2.4.1|1192| 42| 42| 1.2 KB| > |2.7.1|1222179| 48740| 48735| 1.21 MB| > Most RESTful API requesters don't know about this after upgraded and their > old queries may cause ResourceManager more GC consuming and slower. Even if > they know this but have no idea to reduce the impact of ResourceManager > except slow down their query frequency. > The patch adding a query parameter "showResourceRequests" to help requesters > who don't need this information to reduce the overhead. In consideration of > compatibility of interface, the default value is true if they don't set the > parameter, so the behaviour is the same as now. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6428) Queue AM limit is not honored in CS always
[ https://issues.apache.org/jira/browse/YARN-6428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036525#comment-16036525 ] Bibin A Chundatt commented on YARN-6428: [~varun_saxena][~naganarasimha...@apache.org]/[~sunilg] Any review comments i need to update. > Queue AM limit is not honored in CS always > --- > > Key: YARN-6428 > URL: https://issues.apache.org/jira/browse/YARN-6428 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: YARN-6428.0001.patch > > > Steps to reproduce > > Setup cluster with 40 GB and 40 vcores with 4 Node managers with 10 GB each. > Configure 100% to default queue as capacity and max am limit as 10 % > Minimum scheduler memory and vcore as 512,1 > *Expected* > AM limit 4096 and 4 vores > *Actual* > AM limit 4096+512 and 4+1 vcore -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6280) Add a query parameter in ResourceManager Cluster Applications REST API to control whether or not returns ResourceRequest
[
https://issues.apache.org/jira/browse/YARN-6280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036496#comment-16036496
]
Lantao Jin commented on YARN-6280:
--
[~sunilg] Correct. And it also could help to illustrate the new parameter
{{deSelects}} as an example.
> Add a query parameter in ResourceManager Cluster Applications REST API to
> control whether or not returns ResourceRequest
>
>
> Key: YARN-6280
> URL: https://issues.apache.org/jira/browse/YARN-6280
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager, restapi
>Affects Versions: 2.7.3
>Reporter: Lantao Jin
>Assignee: Lantao Jin
> Attachments: YARN-6280.001.patch, YARN-6280.002.patch,
> YARN-6280.003.patch, YARN-6280.004.patch, YARN-6280.005.patch,
> YARN-6280.006.patch, YARN-6280.007.patch, YARN-6280.008.patch,
> YARN-6280.009.patch, YARN-6280.010.patch
>
>
> Begin from v2.7, the ResourceManager Cluster Applications REST API returns
> ResourceRequest list. It's a very large construction in AppInfo.
> As a test, we use below URI to query only 2 results:
> http:// address:port>/ws/v1/cluster/apps?states=running,accepted&limit=2
> The results are very different:
> ||Hadoop version|Total Character|Total Word|Total Lines|Size||
> |2.4.1|1192| 42| 42| 1.2 KB|
> |2.7.1|1222179| 48740| 48735| 1.21 MB|
> Most RESTful API requesters don't know about this after upgraded and their
> old queries may cause ResourceManager more GC consuming and slower. Even if
> they know this but have no idea to reduce the impact of ResourceManager
> except slow down their query frequency.
> The patch adding a query parameter "showResourceRequests" to help requesters
> who don't need this information to reduce the overhead. In consideration of
> compatibility of interface, the default value is true if they don't set the
> parameter, so the behaviour is the same as now.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6688) Add client interface to know default queue for user
[ https://issues.apache.org/jira/browse/YARN-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036486#comment-16036486 ] Naganarasimha G R commented on YARN-6688: - Thanks [~bibinchundatt] for raising this issue, With complex user/group mappings i presume it would be definitely useful to pre determine which queue is picked by default when an app is submitted. > Add client interface to know default queue for user > --- > > Key: YARN-6688 > URL: https://issues.apache.org/jira/browse/YARN-6688 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > > Currently user gets to know queue placement only once application is > accepted. Provide an option for client/user to know default queue for user. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6688) Add client interface to know default queue for user
[ https://issues.apache.org/jira/browse/YARN-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-6688: Issue Type: Improvement (was: Bug) > Add client interface to know default queue for user > --- > > Key: YARN-6688 > URL: https://issues.apache.org/jira/browse/YARN-6688 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > > Currently user gets to know queue placement only once application is > accepted. Provide an option for client/user to know default queue for user. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-1471) The SLS simulator is not running the preemption policy for CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated YARN-1471: -- Attachment: YARN-1471-branch-2.8.patch > The SLS simulator is not running the preemption policy for CapacityScheduler > > > Key: YARN-1471 > URL: https://issues.apache.org/jira/browse/YARN-1471 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Carlo Curino >Assignee: Carlo Curino >Priority: Minor > Labels: release-blocker > Fix For: 3.0.0-alpha1 > > Attachments: SLSCapacityScheduler.java, YARN-1471.2.patch, > YARN-1471-branch-2.7.4.patch, YARN-1471-branch-2.8.patch, > YARN-1471-branch-2.patch, YARN-1471.patch, YARN-1471.patch > > > The simulator does not run the ProportionalCapacityPreemptionPolicy monitor. > This is because the policy needs to interact with a CapacityScheduler, and > the wrapping done by the simulator breaks this. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-1471) The SLS simulator is not running the preemption policy for CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated YARN-1471: -- Attachment: YARN-1471-branch-2.7.4.patch > The SLS simulator is not running the preemption policy for CapacityScheduler > > > Key: YARN-1471 > URL: https://issues.apache.org/jira/browse/YARN-1471 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Carlo Curino >Assignee: Carlo Curino >Priority: Minor > Labels: release-blocker > Fix For: 3.0.0-alpha1 > > Attachments: SLSCapacityScheduler.java, YARN-1471.2.patch, > YARN-1471-branch-2.7.4.patch, YARN-1471-branch-2.patch, YARN-1471.patch, > YARN-1471.patch > > > The simulator does not run the ProportionalCapacityPreemptionPolicy monitor. > This is because the policy needs to interact with a CapacityScheduler, and > the wrapping done by the simulator breaks this. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-1471) The SLS simulator is not running the preemption policy for CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated YARN-1471: -- Attachment: (was: YARN-1471-branch-2.7.4.patch.001) > The SLS simulator is not running the preemption policy for CapacityScheduler > > > Key: YARN-1471 > URL: https://issues.apache.org/jira/browse/YARN-1471 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Carlo Curino >Assignee: Carlo Curino >Priority: Minor > Labels: release-blocker > Fix For: 3.0.0-alpha1 > > Attachments: SLSCapacityScheduler.java, YARN-1471.2.patch, > YARN-1471-branch-2.patch, YARN-1471.patch, YARN-1471.patch > > > The simulator does not run the ProportionalCapacityPreemptionPolicy monitor. > This is because the policy needs to interact with a CapacityScheduler, and > the wrapping done by the simulator breaks this. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6688) Add client interface to know default queue for user
[ https://issues.apache.org/jira/browse/YARN-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036378#comment-16036378 ] Varun Saxena commented on YARN-6688: Can you elaborate upon the potential use case? > Add client interface to know default queue for user > --- > > Key: YARN-6688 > URL: https://issues.apache.org/jira/browse/YARN-6688 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > > Currently user gets to know queue placement only once application is > accepted. Provide an option for client/user to know default queue for user. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5988) RM unable to start in secure setup
[ https://issues.apache.org/jira/browse/YARN-5988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated YARN-5988: -- Labels: (was: release-blocker) > RM unable to start in secure setup > -- > > Key: YARN-5988 > URL: https://issues.apache.org/jira/browse/YARN-5988 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.9.0, 3.0.0-alpha1 >Reporter: Ajith S >Assignee: Ajith S >Priority: Blocker > Fix For: 2.9.0, 2.7.4, 3.0.0-alpha2, 2.8.2 > > Attachments: hadoop-secureuser-resourcemanager-vm1.log, > YARN-5988.01.patch, YARN-5988.02.patch, YARN-5988.03.patch, > YARN-5988.04.patch, YARN-5988.05.patch, YARN-5988-branch-2.7.05.patch, > YARN-5988-branch-2.8.05.patch > > > When CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHORIZATION=true > RM is unable to start -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5988) RM unable to start in secure setup
[ https://issues.apache.org/jira/browse/YARN-5988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036377#comment-16036377 ] Konstantin Shvachko commented on YARN-5988: --- OK, great! Could you please replace my patches with the ones committed under YARN-6664. > RM unable to start in secure setup > -- > > Key: YARN-5988 > URL: https://issues.apache.org/jira/browse/YARN-5988 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.9.0, 3.0.0-alpha1 >Reporter: Ajith S >Assignee: Ajith S >Priority: Blocker > Labels: release-blocker > Fix For: 2.9.0, 2.7.4, 3.0.0-alpha2, 2.8.2 > > Attachments: hadoop-secureuser-resourcemanager-vm1.log, > YARN-5988.01.patch, YARN-5988.02.patch, YARN-5988.03.patch, > YARN-5988.04.patch, YARN-5988.05.patch, YARN-5988-branch-2.7.05.patch, > YARN-5988-branch-2.8.05.patch > > > When CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHORIZATION=true > RM is unable to start -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6458) Use yarn package manager to lock down dependency versions for new web UI
[ https://issues.apache.org/jira/browse/YARN-6458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036337#comment-16036337 ] Hudson commented on YARN-6458: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11821 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11821/]) YARN-6458. Use yarn package manager to lock down dependency versions for (sunilg: rev 46f7e91980b2c7a0fec749b5ce833be49d86c061) * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/WEB-INF/wro.xml * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/README.md * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/bower-shrinkwrap.json * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/package.json * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/pom.xml * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/ember-cli-build.js * (add) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/yarn.lock * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/.bowerrc > Use yarn package manager to lock down dependency versions for new web UI > > > Key: YARN-6458 > URL: https://issues.apache.org/jira/browse/YARN-6458 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Fix For: 3.0.0-alpha4 > > Attachments: YARN-6458.1.patch, YARN-6458.2.patch, YARN-6458.3.patch > > > As we use semver to denote dependency version, every time a new build is > made, the latest available version of the dependency would be downloaded. > This affects the reliability of the UI build. Hence we must lockdown the > dependencies. > Lockdown must happen in both the package managers used by the UI - NPM & > Bower. > Yarn: > Replace NPM with Yarn. Yarn is a package manager developed to solve this > issue and many more. It also enables offline build. > Bower: > Bower shrinkwrap resolver plugin can be used to lock the dependency versions. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-6458) Use yarn package manager to lock down dependency versions for new web UI
[ https://issues.apache.org/jira/browse/YARN-6458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-6458: -- Summary: Use yarn package manager to lock down dependency versions for new web UI (was: New Yarn UI: Lock down dependency versions) > Use yarn package manager to lock down dependency versions for new web UI > > > Key: YARN-6458 > URL: https://issues.apache.org/jira/browse/YARN-6458 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: YARN-6458.1.patch, YARN-6458.2.patch, YARN-6458.3.patch > > > As we use semver to denote dependency version, every time a new build is > made, the latest available version of the dependency would be downloaded. > This affects the reliability of the UI build. Hence we must lockdown the > dependencies. > Lockdown must happen in both the package managers used by the UI - NPM & > Bower. > Yarn: > Replace NPM with Yarn. Yarn is a package manager developed to solve this > issue and many more. It also enables offline build. > Bower: > Bower shrinkwrap resolver plugin can be used to lock the dependency versions. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6280) Add a query parameter in ResourceManager Cluster Applications REST API to control whether or not returns ResourceRequest
[
https://issues.apache.org/jira/browse/YARN-6280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036311#comment-16036311
]
Sunil G commented on YARN-6280:
---
Code changes are looking fine for me.
In ResourceManagerRest.md file, I could see that {{resourceRequests}} is added
as example in 3 places. I suppose it was not needed for this patch, but it was
fundamentally missing from earlier itself, correct?
> Add a query parameter in ResourceManager Cluster Applications REST API to
> control whether or not returns ResourceRequest
>
>
> Key: YARN-6280
> URL: https://issues.apache.org/jira/browse/YARN-6280
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager, restapi
>Affects Versions: 2.7.3
>Reporter: Lantao Jin
>Assignee: Lantao Jin
> Attachments: YARN-6280.001.patch, YARN-6280.002.patch,
> YARN-6280.003.patch, YARN-6280.004.patch, YARN-6280.005.patch,
> YARN-6280.006.patch, YARN-6280.007.patch, YARN-6280.008.patch,
> YARN-6280.009.patch, YARN-6280.010.patch
>
>
> Begin from v2.7, the ResourceManager Cluster Applications REST API returns
> ResourceRequest list. It's a very large construction in AppInfo.
> As a test, we use below URI to query only 2 results:
> http:// address:port>/ws/v1/cluster/apps?states=running,accepted&limit=2
> The results are very different:
> ||Hadoop version|Total Character|Total Word|Total Lines|Size||
> |2.4.1|1192| 42| 42| 1.2 KB|
> |2.7.1|1222179| 48740| 48735| 1.21 MB|
> Most RESTful API requesters don't know about this after upgraded and their
> old queries may cause ResourceManager more GC consuming and slower. Even if
> they know this but have no idea to reduce the impact of ResourceManager
> except slow down their query frequency.
> The patch adding a query parameter "showResourceRequests" to help requesters
> who don't need this information to reduce the overhead. In consideration of
> compatibility of interface, the default value is true if they don't set the
> parameter, so the behaviour is the same as now.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-6688) Add client interface to know default queue for user
Bibin A Chundatt created YARN-6688: -- Summary: Add client interface to know default queue for user Key: YARN-6688 URL: https://issues.apache.org/jira/browse/YARN-6688 Project: Hadoop YARN Issue Type: Bug Reporter: Bibin A Chundatt Assignee: Bibin A Chundatt Currently user gets to know queue placement only once application is accepted. Provide an option for client/user to know default queue for user. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-5988) RM unable to start in secure setup
[ https://issues.apache.org/jira/browse/YARN-5988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-5988: -- Target Version/s: 3.0.0-alpha2, 2.9.0, 2.7.4 (was: 2.9.0, 2.7.4, 3.0.0-alpha2) Fix Version/s: 2.8.2 2.7.4 > RM unable to start in secure setup > -- > > Key: YARN-5988 > URL: https://issues.apache.org/jira/browse/YARN-5988 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.9.0, 3.0.0-alpha1 >Reporter: Ajith S >Assignee: Ajith S >Priority: Blocker > Labels: release-blocker > Fix For: 2.9.0, 2.7.4, 3.0.0-alpha2, 2.8.2 > > Attachments: hadoop-secureuser-resourcemanager-vm1.log, > YARN-5988.01.patch, YARN-5988.02.patch, YARN-5988.03.patch, > YARN-5988.04.patch, YARN-5988.05.patch, YARN-5988-branch-2.7.05.patch, > YARN-5988-branch-2.8.05.patch > > > When CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHORIZATION=true > RM is unable to start -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6664) Backport YARN-5988 to branch-2.8, branch-2.7
[ https://issues.apache.org/jira/browse/YARN-6664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036214#comment-16036214 ] Sunil G commented on YARN-6664: --- Thanks [~rohithsharma] for the patches. Committed to branch-2.7 and branch-2.8. I will update fixed version in YARN-5988 and will close this ticket as Done. > Backport YARN-5988 to branch-2.8, branch-2.7 > > > Key: YARN-6664 > URL: https://issues.apache.org/jira/browse/YARN-6664 > Project: Hadoop YARN > Issue Type: Task >Reporter: Rohith Sharma K S >Assignee: Rohith Sharma K S > Attachments: YARN-5988-branch-2.7.0001.patch, > YARN-5988-branch-2.8.0001.patch, YARN-5988-branch-2.8.0001.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6664) Backport YARN-5988 to branch-2.8, branch-2.7
[
https://issues.apache.org/jira/browse/YARN-6664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16036211#comment-16036211
]
Hadoop QA commented on YARN-6664:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
31s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
2s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} branch-2.8 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} branch-2.8 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s{color} | {color:green} branch-2.8 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} branch-2.8 passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} branch-2.8 passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed with JDK v1.8.0_131 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed with JDK v1.7.0_131 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 75m 42s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed with JDK
v1.7.0_131. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}169m 12s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_131 Failed junit tests |
hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| | hadoop.yarn.server.resourcemanager.TestRMHA |
| | hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| JDK v1.7.0_131 Failed junit tests |
hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| | hadoop.yarn.server.resourcemanager.TestRMHA |
| | hadoop.yarn.server.resourcemanager.TestClientRMTokens |
| | hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:d946387 |
| JIRA Issue | YARN-6664 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12871142/YARN-5988-branch-2.8.0001.patch
|
| Optional Tests | asf
