[jira] [Commented] (YARN-10397) SchedulerRequest should be forwarded to scheduler if custom scheduler supports placement constraints
[ https://issues.apache.org/jira/browse/YARN-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192941#comment-17192941 ] Bilwa S T commented on YARN-10397: -- Thanks [~brahmareddy] > SchedulerRequest should be forwarded to scheduler if custom scheduler > supports placement constraints > > > Key: YARN-10397 > URL: https://issues.apache.org/jira/browse/YARN-10397 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Minor > Fix For: 3.4.0, 3.3.1 > > Attachments: YARN-10397.001.patch, YARN-10397.002.patch > > > Currently only CapacityScheduler supports placement constraints so request > gets forwarded only for capacityScheduler. Below exception will be thrown if > custom scheduler supports placement constraint > {code:java} > if (request.getSchedulingRequests() != null > && !request.getSchedulingRequests().isEmpty()) { > if (!(scheduler instanceof CapacityScheduler)) { > String message = "Found non empty SchedulingRequest of " > + "AllocateRequest for application=" + appAttemptId.toString() > + ", however the configured scheduler=" > + scheduler.getClass().getCanonicalName() > + " cannot handle placement constraints, rejecting this " > + "allocate operation"; > LOG.warn(message); > throw new YarnException(message); > } > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10397) SchedulerRequest should be forwarded to scheduler if custom scheduler supports placement constraints
[ https://issues.apache.org/jira/browse/YARN-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17192799#comment-17192799 ] Brahma Reddy Battula commented on YARN-10397: - +1, Going to commit shortly. > SchedulerRequest should be forwarded to scheduler if custom scheduler > supports placement constraints > > > Key: YARN-10397 > URL: https://issues.apache.org/jira/browse/YARN-10397 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Minor > Attachments: YARN-10397.001.patch, YARN-10397.002.patch > > > Currently only CapacityScheduler supports placement constraints so request > gets forwarded only for capacityScheduler. Below exception will be thrown if > custom scheduler supports placement constraint > {code:java} > if (request.getSchedulingRequests() != null > && !request.getSchedulingRequests().isEmpty()) { > if (!(scheduler instanceof CapacityScheduler)) { > String message = "Found non empty SchedulingRequest of " > + "AllocateRequest for application=" + appAttemptId.toString() > + ", however the configured scheduler=" > + scheduler.getClass().getCanonicalName() > + " cannot handle placement constraints, rejecting this " > + "allocate operation"; > LOG.warn(message); > throw new YarnException(message); > } > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10397) SchedulerRequest should be forwarded to scheduler if custom scheduler supports placement constraints
[ https://issues.apache.org/jira/browse/YARN-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180852#comment-17180852 ] Hadoop QA commented on YARN-10397: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 39s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m 0s{color} | {color:green} No case conflicting files found. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s{color} | {color:green} trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s{color} | {color:green} trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 43s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 53s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 15m 24s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s{color} | {color:green} trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s{color} | {color:green} trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} | | {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m 42s{color} | {color:blue} Used deprecated FindBugs config; considering switching to SpotBugs. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 40s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 46s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s{color} | {color:green} the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s{color} | {color:green} the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 23s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s{color} | {color:green} the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s{color} | {color:green} the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 43s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green}104m 11s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch passed. {color} | |
[jira] [Commented] (YARN-10397) SchedulerRequest should be forwarded to scheduler if custom scheduler supports placement constraints
[ https://issues.apache.org/jira/browse/YARN-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180784#comment-17180784 ] Íñigo Goiri commented on YARN-10397: +1 on [^YARN-10397.002.patch]. > SchedulerRequest should be forwarded to scheduler if custom scheduler > supports placement constraints > > > Key: YARN-10397 > URL: https://issues.apache.org/jira/browse/YARN-10397 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Minor > Attachments: YARN-10397.001.patch, YARN-10397.002.patch > > > Currently only CapacityScheduler supports placement constraints so request > gets forwarded only for capacityScheduler. Below exception will be thrown if > custom scheduler supports placement constraint > {code:java} > if (request.getSchedulingRequests() != null > && !request.getSchedulingRequests().isEmpty()) { > if (!(scheduler instanceof CapacityScheduler)) { > String message = "Found non empty SchedulingRequest of " > + "AllocateRequest for application=" + appAttemptId.toString() > + ", however the configured scheduler=" > + scheduler.getClass().getCanonicalName() > + " cannot handle placement constraints, rejecting this " > + "allocate operation"; > LOG.warn(message); > throw new YarnException(message); > } > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10397) SchedulerRequest should be forwarded to scheduler if custom scheduler supports placement constraints
[ https://issues.apache.org/jira/browse/YARN-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180774#comment-17180774 ] Bilwa S T commented on YARN-10397: -- Thanks [~elgoiri] for reviewing this. I have updated javadoc . Yes this is covered by UT TestCapacitySchedulerSchedulingRequestUpdate. This testcase checks if capacityscheduler supports placement constraint or not. bq. BTW, I'm guessing you are using your own scheduler that supports this? Yes we have our own scheduler which supports placement constraints. > SchedulerRequest should be forwarded to scheduler if custom scheduler > supports placement constraints > > > Key: YARN-10397 > URL: https://issues.apache.org/jira/browse/YARN-10397 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Minor > Attachments: YARN-10397.001.patch, YARN-10397.002.patch > > > Currently only CapacityScheduler supports placement constraints so request > gets forwarded only for capacityScheduler. Below exception will be thrown if > custom scheduler supports placement constraint > {code:java} > if (request.getSchedulingRequests() != null > && !request.getSchedulingRequests().isEmpty()) { > if (!(scheduler instanceof CapacityScheduler)) { > String message = "Found non empty SchedulingRequest of " > + "AllocateRequest for application=" + appAttemptId.toString() > + ", however the configured scheduler=" > + scheduler.getClass().getCanonicalName() > + " cannot handle placement constraints, rejecting this " > + "allocate operation"; > LOG.warn(message); > throw new YarnException(message); > } > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10397) SchedulerRequest should be forwarded to scheduler if custom scheduler supports placement constraints
[ https://issues.apache.org/jira/browse/YARN-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17180735#comment-17180735 ] Íñigo Goiri commented on YARN-10397: It would be nice to add a javadoc placementConstraintEnabled() in both the abstract scheduler and the capacity scheduler specifying this. BTW, is this covered by any unit test? BTW, I'm guessing you are using your own scheduler that supports this? > SchedulerRequest should be forwarded to scheduler if custom scheduler > supports placement constraints > > > Key: YARN-10397 > URL: https://issues.apache.org/jira/browse/YARN-10397 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Minor > Attachments: YARN-10397.001.patch > > > Currently only CapacityScheduler supports placement constraints so request > gets forwarded only for capacityScheduler. Below exception will be thrown if > custom scheduler supports placement constraint > {code:java} > if (request.getSchedulingRequests() != null > && !request.getSchedulingRequests().isEmpty()) { > if (!(scheduler instanceof CapacityScheduler)) { > String message = "Found non empty SchedulingRequest of " > + "AllocateRequest for application=" + appAttemptId.toString() > + ", however the configured scheduler=" > + scheduler.getClass().getCanonicalName() > + " cannot handle placement constraints, rejecting this " > + "allocate operation"; > LOG.warn(message); > throw new YarnException(message); > } > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10397) SchedulerRequest should be forwarded to scheduler if custom scheduler supports placement constraints
[ https://issues.apache.org/jira/browse/YARN-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178001#comment-17178001 ] Hadoop QA commented on YARN-10397: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m 11s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m 0s{color} | {color:green} No case conflicting files found. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m 0s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 17s{color} | {color:green} trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 46s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 4s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 20m 49s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s{color} | {color:green} trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s{color} | {color:green} trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} | | {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m 46s{color} | {color:blue} Used deprecated FindBugs config; considering switching to SpotBugs. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 43s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 46s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s{color} | {color:green} the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s{color} | {color:green} the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 45s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 15m 32s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s{color} | {color:green} the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s{color} | {color:green} the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 59s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 57s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} | |
[jira] [Commented] (YARN-10397) SchedulerRequest should be forwarded to scheduler if custom scheduler supports placement constraints
[ https://issues.apache.org/jira/browse/YARN-10397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177834#comment-17177834 ] Bilwa S T commented on YARN-10397: -- cc [~leftnoteasy] [~prabhujoseph] > SchedulerRequest should be forwarded to scheduler if custom scheduler > supports placement constraints > > > Key: YARN-10397 > URL: https://issues.apache.org/jira/browse/YARN-10397 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Major > Attachments: YARN-10397.001.patch > > > Currently only CapacityScheduler supports placement constraints so request > gets forwarded only for capacityScheduler. Below exception will be thrown if > custom scheduler supports placement constraint > {code:java} > if (request.getSchedulingRequests() != null > && !request.getSchedulingRequests().isEmpty()) { > if (!(scheduler instanceof CapacityScheduler)) { > String message = "Found non empty SchedulingRequest of " > + "AllocateRequest for application=" + appAttemptId.toString() > + ", however the configured scheduler=" > + scheduler.getClass().getCanonicalName() > + " cannot handle placement constraints, rejecting this " > + "allocate operation"; > LOG.warn(message); > throw new YarnException(message); > } > } > {code} > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org