[jira] [Resolved] (YARN-10971) Continuous scheduling reduces invalid calculations
[ https://issues.apache.org/jira/browse/YARN-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangminghai resolved YARN-10971. - Resolution: Resolved > Continuous scheduling reduces invalid calculations > -- > > Key: YARN-10971 > URL: https://issues.apache.org/jira/browse/YARN-10971 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.7.2 >Reporter: Jiangminghai >Priority: Minor > Fix For: 2.7.2 > > > In FairScheduler#continuousSchedulingAttempt(), nodes has been sorted by > available resources，iterate all nodes in continuous scheduling. > Should break the loop, when > *Resources.fitsIn(minimumAllocation,node.getUnallocatedResource()) == false*. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10971) Continuous scheduling reduces invalid calculations
[
https://issues.apache.org/jira/browse/YARN-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425913#comment-17425913
]
Hadoop QA commented on YARN-10971:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 10s{color}
| {color:red}{color} | {color:red} YARN-10971 does not apply to trunk. Rebase
required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for
help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-10971 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13034676/YARN-10971.0.patch |
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/1221/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Continuous scheduling reduces invalid calculations
> --
>
> Key: YARN-10971
> URL: https://issues.apache.org/jira/browse/YARN-10971
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
>Affects Versions: 2.7.2
>Reporter: Jiangminghai
>Priority: Minor
> Fix For: 2.7.2
>
> Attachments: YARN-10971.0.patch
>
>
> In FairScheduler#continuousSchedulingAttempt(), nodes has been sorted by
> available resources，iterate all nodes in continuous scheduling.
> Should break the loop, when
> *Resources.fitsIn(minimumAllocation,node.getUnallocatedResource()) == false*.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10971) Continuous scheduling reduces invalid calculations
[
https://issues.apache.org/jira/browse/YARN-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425955#comment-17425955
]
Hadoop QA commented on YARN-10971:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m
11s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} || ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} No case conflicting files
found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch does not contain any
@author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red}{color} | {color:red} The patch doesn't appear to
include any new or modified tests. Please justify why no new tests are needed
for this patch. Also please list what manual steps were performed to verify
this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
18s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green}{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 28s{color} | {color:green}{color} | {color:green} branch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green}{color} | {color:green} trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 18m
56s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are
enabled, using SpotBugs. {color} |
| {color:green}+1{color} | {color:green} spotbugs {color} | {color:green} 1m
49s{color} | {color:green}{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} || ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
47s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
50s{color} | {color:green}{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green}{color} | {color:green} The patch has no whitespace
issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 22s{color} | {color:green}{color} | {color:green} patch has no errors when
building and testing our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
42s{color} | {color:green}{color} | {color:green} the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 {color} |
|
[jira] [Updated] (YARN-10971) Continuous scheduling reduces invalid calculations
[ https://issues.apache.org/jira/browse/YARN-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangminghai updated YARN-10971: Attachment: (was: YARN-10971.0.patch) > Continuous scheduling reduces invalid calculations > -- > > Key: YARN-10971 > URL: https://issues.apache.org/jira/browse/YARN-10971 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.7.2 >Reporter: Jiangminghai >Priority: Minor > > In FairScheduler#continuousSchedulingAttempt(), nodes has been sorted by > available resources，iterate all nodes in continuous scheduling. > Should break the loop, when > *Resources.fitsIn(minimumAllocation,node.getUnallocatedResource()) == false*. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Reopened] (YARN-10971) Continuous scheduling reduces invalid calculations
[ https://issues.apache.org/jira/browse/YARN-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangminghai reopened YARN-10971: - > Continuous scheduling reduces invalid calculations > -- > > Key: YARN-10971 > URL: https://issues.apache.org/jira/browse/YARN-10971 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.7.2 >Reporter: Jiangminghai >Priority: Minor > Fix For: 2.7.2 > > > In FairScheduler#continuousSchedulingAttempt(), nodes has been sorted by > available resources，iterate all nodes in continuous scheduling. > Should break the loop, when > *Resources.fitsIn(minimumAllocation,node.getUnallocatedResource()) == false*. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10971) Continuous scheduling reduces invalid calculations
[ https://issues.apache.org/jira/browse/YARN-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangminghai updated YARN-10971: Attachment: (was: YARN-10971.0.patch) > Continuous scheduling reduces invalid calculations > -- > > Key: YARN-10971 > URL: https://issues.apache.org/jira/browse/YARN-10971 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.7.2 >Reporter: Jiangminghai >Priority: Minor > Fix For: 2.7.2 > > > In FairScheduler#continuousSchedulingAttempt(), nodes has been sorted by > available resources，iterate all nodes in continuous scheduling. > Should break the loop, when > *Resources.fitsIn(minimumAllocation,node.getUnallocatedResource()) == false*. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Reopened] (YARN-10971) Continuous scheduling reduces invalid calculations
[ https://issues.apache.org/jira/browse/YARN-10971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangminghai reopened YARN-10971: - > Continuous scheduling reduces invalid calculations > -- > > Key: YARN-10971 > URL: https://issues.apache.org/jira/browse/YARN-10971 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler >Affects Versions: 2.7.2 >Reporter: Jiangminghai >Priority: Minor > Fix For: 2.7.2 > > Attachments: YARN-10971.0.patch > > > In FairScheduler#continuousSchedulingAttempt(), nodes has been sorted by > available resources，iterate all nodes in continuous scheduling. > Should break the loop, when > *Resources.fitsIn(minimumAllocation,node.getUnallocatedResource()) == false*. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8936) Bump up Atsv2 hbase versions
[
https://issues.apache.org/jira/browse/YARN-8936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425365#comment-17425365
]
Masatake Iwasaki commented on YARN-8936:
{quote}
Ideally we could have created one more to iira to backport this so that release
notes can be proper as we already given release after this resolved.
{quote}
[~brahmareddy] [releasedocmaker of
Yetus|https://github.com/apache/yetus/blob/rel/0.13.0/releasedocmaker/src/main/python/releasedocmaker/__init__.py#L341-L343]
picks this JIRA if "Fix Version/s:" contains the target version. Adding values
to the "Fix Version/s:" field should not affect RELEASENOTE/CHANGELOG of other
releases?
{noformat}
$ git checkout branch-3.2.3
$ mvn clean install -DskipTests -DskipShade
$ mvn site -Preleasedocs
$ grep -r YARN-8936 ./hadoop-common-project/hadoop-common/target/site/release
./hadoop-common-project/hadoop-common/target/site/release/3.2.3/CHANGELOG.3.2.3.html: https://issues.apache.org/jira/browse/YARN-8936;>YARN-8936
{noformat}
> Bump up Atsv2 hbase versions
>
>
> Key: YARN-8936
> URL: https://issues.apache.org/jira/browse/YARN-8936
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.2.0, 3.1.2, 3.3.0
>Reporter: Rohith Sharma K S
>Assignee: Vrushali C
>Priority: Major
> Fix For: 3.3.0, 3.2.3
>
> Attachments: YARN-8936-yarn-7055.0001.patch, YARN-8936.0001.patch,
> YARN-8936.0002.patch, YARN-8936.0003.patch
>
>
> Hadoop trunk uses hbase.two.version as 2.0.0-beta-1. HBase has release stable
> Hbase-2.0.2 version and this could be used in Hadoop-3.3/3.2/3.1 branches.
> cc:/ [~vrushalic]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-10962) Do not extend from CapacitySchedulerTestBase when not needed
[ https://issues.apache.org/jira/browse/YARN-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gergely Pollák resolved YARN-10962. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Target Version/s: 3.4.0 Resolution: Fixed > Do not extend from CapacitySchedulerTestBase when not needed > > > Key: YARN-10962 > URL: https://issues.apache.org/jira/browse/YARN-10962 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Tamas Domok >Assignee: Tamas Domok >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > > The following classes extends CapacitySchedulerTestBase, but it is > unnecessary. > - TestCapacitySchedulerMultiNodes > - TestCapacitySchedulerMultiNodesWithPreemption > - TestCapacitySchedulerSchedulingRequestUpdate > - TestCapacitySchedulerWorkflowPriorityMapping > The helper methods could be static, so inheritance is not needed here. > > Later the TestCapacityScheduler can be split by features, where the > CapacitySchedulerTestBase will come handy. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10962) Do not extend from CapacitySchedulerTestBase when not needed
[ https://issues.apache.org/jira/browse/YARN-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425587#comment-17425587 ] Gergely Pollák commented on YARN-10962: --- [~tdomok] thank you for working on this, LGTM+1, merged to trunk. > Do not extend from CapacitySchedulerTestBase when not needed > > > Key: YARN-10962 > URL: https://issues.apache.org/jira/browse/YARN-10962 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Tamas Domok >Assignee: Tamas Domok >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > > The following classes extends CapacitySchedulerTestBase, but it is > unnecessary. > - TestCapacitySchedulerMultiNodes > - TestCapacitySchedulerMultiNodesWithPreemption > - TestCapacitySchedulerSchedulingRequestUpdate > - TestCapacitySchedulerWorkflowPriorityMapping > The helper methods could be static, so inheritance is not needed here. > > Later the TestCapacityScheduler can be split by features, where the > CapacitySchedulerTestBase will come handy. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9009) Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs
[
https://issues.apache.org/jira/browse/YARN-9009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425502#comment-17425502
]
Hadoop QA commented on YARN-9009:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 21s{color}
| | {color:red} https://github.com/apache/hadoop/pull/438 does not apply to
trunk. Rebase required? Wrong Branch? See
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| GITHUB PR | https://github.com/apache/hadoop/pull/438 |
| JIRA Issue | YARN-9009 |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-438/2/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs
> ---
>
> Key: YARN-9009
> URL: https://issues.apache.org/jira/browse/YARN-9009
> Project: Hadoop YARN
> Issue Type: Bug
> Environment: Ubuntu 18.04
> java version "1.8.0_181"
> Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
>
> Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe;
> 2018-06-17T13:33:14-05:00)
>Reporter: OrDTesters
>Assignee: OrDTesters
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.0.4, 3.1.2, 3.3.0, 3.2.1
>
> Attachments: YARN-9009-trunk-001.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In TestEntityGroupFSTimelineStore, testCleanLogs fails when run after
> testMoveToDone.
> testCleanLogs fails because testMoveToDone moves a file into the same
> directory that testCleanLogs cleans, causing testCleanLogs to clean 3 files,
> instead of 2 as testCleanLogs expects.
> To fix the failure of testCleanLogs, we can delete the file after the file is
> moved by testMoveToDone.
> Pull request link: [https://github.com/apache/hadoop/pull/438]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10953) Make CapacityScheduler#getOrCreateQueueFromPlacementContext easier to comprehend
[ https://issues.apache.org/jira/browse/YARN-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425624#comment-17425624 ] Gergely Pollák commented on YARN-10953: --- [~gandras] thank you for working on this, LGTM+1, merged to trunk > Make CapacityScheduler#getOrCreateQueueFromPlacementContext easier to > comprehend > > > Key: YARN-10953 > URL: https://issues.apache.org/jira/browse/YARN-10953 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Szilard Nemeth >Assignee: Andras Gyori >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > 1. Most of the method body is wrapped in an if-statement that checks if the > queue is null. We could negate this and return immediately if the queue != > null, so we don't need a large if statement. > 2. Similarly in that large if body, there's a check for > fallbackContext.hasParentQueue(). If it's true, we are having yet another > large if-body. We should also negate this condition and return immediately if > it's false. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-10953) Make CapacityScheduler#getOrCreateQueueFromPlacementContext easier to comprehend
[ https://issues.apache.org/jira/browse/YARN-10953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gergely Pollák resolved YARN-10953. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Target Version/s: 3.4.0 Resolution: Resolved > Make CapacityScheduler#getOrCreateQueueFromPlacementContext easier to > comprehend > > > Key: YARN-10953 > URL: https://issues.apache.org/jira/browse/YARN-10953 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Szilard Nemeth >Assignee: Andras Gyori >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > 1. Most of the method body is wrapped in an if-statement that checks if the > queue is null. We could negate this and return immediately if the queue != > null, so we don't need a large if statement. > 2. Similarly in that large if body, there's a check for > fallbackContext.hasParentQueue(). If it's true, we are having yet another > large if-body. We should also negate this condition and return immediately if > it's false. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-9975) Support proxy ACL user for CapacityScheduler
[
https://issues.apache.org/jira/browse/YARN-9975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Payne resolved YARN-9975.
--
Resolution: Duplicate
I'm closing this as a dup of YARN-1115. Please reopen if you disagree.
> Support proxy ACL user for CapacityScheduler
>
>
> Key: YARN-9975
> URL: https://issues.apache.org/jira/browse/YARN-9975
> Project: Hadoop YARN
> Issue Type: New Feature
>Reporter: zhoukang
>Assignee: zhoukang
>Priority: Major
>
> As commented in YARN-9698.
> I will open a new jira for the proxy user feature.
> The background is that we have long running sql thriftserver for many users:
> {quote}{{user->sql proxy-> sql thriftserver}}{quote}
> But we do not have keytab for all users on 'sql proxy'. We just use a super
> user like 'sql_prc' to submit the 'sql thriftserver' application. To support
> this we should change the scheduler to support proxy user acl
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10934) LeafQueue activateApplications NPE
[ https://issues.apache.org/jira/browse/YARN-10934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425728#comment-17425728 ] Gergely Pollák commented on YARN-10934: --- [~luoyuan] thank you for reporting the issue and [~bteke] thank you for fixing it. LGTM+1 (Binding), merged to trunk. > LeafQueue activateApplications NPE > -- > > Key: YARN-10934 > URL: https://issues.apache.org/jira/browse/YARN-10934 > Project: Hadoop YARN > Issue Type: Bug > Components: RM >Affects Versions: 3.3.1 >Reporter: Yuan Luo >Assignee: Benjamin Teke >Priority: Major > Labels: pull-request-available > Attachments: RM-capacity-scheduler.xml, RM-yarn-site.xml > > Time Spent: 1h > Remaining Estimate: 0h > > Our prod Yarn cluster is hadoop version 3.3.1 , we changed > DefaultResourceCalculator -> DominantResourceCalculator and restart RM, then > our RM crashed, the Exception stack like below. I think this is a serious > bug and hope someone can follow up and fix it. > 2021-08-30 21:00:59,114 ERROR event.EventDispatcher > (MarkerIgnoringBase.java:error(159)) - Error in handling event type > APP_ATTEMPT_REMOVED to the Event Dispatcher > java.lang.NullPointerException > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.activateApplications(LeafQueue.java:868) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.removeApplicationAttempt(LeafQueue.java:1014) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.finishApplicationAttempt(LeafQueue.java:972) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.doneApplicationAttempt(CapacityScheduler.java:1188) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:1904) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:171) > at > org.apache.hadoop.yarn.event.EventDispatcher$EventProcessor.run(EventDispatcher.java:79) > at java.base/java.lang.Thread.run(Thread.java:834) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
