[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063785#comment-14063785 ] Craig Welch commented on YARN-1198: --- [~jlowe] Right, this isn't a replacement for the

[jira] [Commented] (YARN-1680) availableResources sent to applicationMaster in heartbeat should exclude blacklistedNodes free memory.

[ https://issues.apache.org/jira/browse/YARN-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064012#comment-14064012 ] Craig Welch commented on YARN-1680: --- [~airbots] did you want to switch over to using a

[jira] [Updated] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-2008: -- Attachment: YARN-2008.1.patch Patch implementing the described behavior... CapacityScheduler may

[jira] [Commented] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14066510#comment-14066510 ] Craig Welch commented on YARN-2008: --- [~airbots] Chen, I put together a patch, with it I

[jira] [Commented] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14066511#comment-14066511 ] Craig Welch commented on YARN-2008: --- [~wangda], can you have a look at this pls? This is

[jira] [Updated] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-2008: -- Attachment: YARN-2008.2.patch Added missed unit test CapacityScheduler may report incorrect

[jira] [Commented] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14066654#comment-14066654 ] Craig Welch commented on YARN-2008: --- The tests seem to pass on my box, I think these are

[jira] [Commented] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14067078#comment-14067078 ] Craig Welch commented on YARN-2008: --- And, the two which failed this time also pass on my

[jira] [Commented] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068024#comment-14068024 ] Craig Welch commented on YARN-2008: --- Hmm, I don't think that we can change headroom to be

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068791#comment-14068791 ] Craig Welch commented on YARN-1198: --- [~wangda] I concur with [~jlowe] and [~airbots] that

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068947#comment-14068947 ] Craig Welch commented on YARN-1198: --- Also - this combined with preemption will be

[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14070818#comment-14070818 ] Craig Welch commented on YARN-1994: --- FYI, took a look at the changes between patch 3 and

[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14070838#comment-14070838 ] Craig Welch commented on YARN-1994: --- -re [~arpitagarwal] comments - [~mipoto] made the

[jira] [Updated] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1994: -- Attachment: YARN-1994.6.patch Expose YARN/MR endpoints on multiple interfaces

[jira] [Assigned] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch reassigned YARN-2008: - Assignee: Craig Welch (was: Chen He) CapacityScheduler may report incorrect queueMaxCap if

[jira] [Commented] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14070859#comment-14070859 ] Craig Welch commented on YARN-2008: --- I believe we've resolved on [YARN-1198] to move

[jira] [Commented] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14070875#comment-14070875 ] Craig Welch commented on YARN-2008: --- [~airbots], -re {quote} and if the application gets

[jira] [Updated] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1994: -- Attachment: YARN-1994.11.patch Fixed bug, YarnConfiguration.getSocketAddr checks in ha cases to see

[jira] [Updated] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1994: -- Attachment: YARN-1994.11.patch Expose YARN/MR endpoints on multiple interfaces

[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14076319#comment-14076319 ] Craig Welch commented on YARN-1994: --- TestAMRestart passes on my box, reattached patch to

[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078048#comment-14078048 ] Craig Welch commented on YARN-1994: --- It used to be the case that the server's

[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078066#comment-14078066 ] Craig Welch commented on YARN-1994: --- On further investigation, the getCanonicalHostname()

[jira] [Updated] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1994: -- Attachment: YARN-1994.12.patch Yup, that change is not needed, nice catch Arpit. Attached is .12

[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078272#comment-14078272 ] Craig Welch commented on YARN-1994: --- Milan At present multi-home only works if all

[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078337#comment-14078337 ] Craig Welch commented on YARN-1994: --- [~mipoto] Trying to understand the scenario here -

[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078685#comment-14078685 ] Craig Welch commented on YARN-1994: --- Hmm, I think I understand the scenario. As I

[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14079371#comment-14079371 ] Craig Welch commented on YARN-1994: --- [~mipoto], it's unfortunate that there is that

[jira] [Updated] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-2008: -- Attachment: YARN-2008.3.patch Added better check for invalid divisor, bounds check for absoluteMaxAvail

[jira] [Updated] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1994: -- Attachment: YARN-1994.13.patch Reversion to earlier logic with updateConnectAddr, with added tests old

[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14079732#comment-14079732 ] Craig Welch commented on YARN-1994: --- I've uploaded the .9 patch (renamed to .13)

[jira] [Updated] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1994: -- Attachment: YARN-1994.14.patch Extension of the .12 patch with updateConnectAddr support for overriding

[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14080297#comment-14080297 ] Craig Welch commented on YARN-1994: --- BTW, .14 patch tested successfully with host names

[jira] [Updated] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1994: -- Attachment: YARN-1994.15.patch Fix javadoc Expose YARN/MR endpoints on multiple interfaces

[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14081025#comment-14081025 ] Craig Welch commented on YARN-1994: --- [~xgong] [~arpitagarwal] [~mipoto] patch .15 should

[jira] [Commented] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14081207#comment-14081207 ] Craig Welch commented on YARN-1994: --- [~mipoto] Can you take a look at the latest patch?

[jira] [Updated] (YARN-1994) Expose YARN/MR endpoints on multiple interfaces

[ https://issues.apache.org/jira/browse/YARN-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1994: -- Attachment: YARN-1994.15-branch2.patch Adding a version of the patch for branch-2, because the one from

[jira] [Updated] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-2008: -- Attachment: YARN-2008.5.patch This time, actually with the additional tests :-) CapacityScheduler

[jira] [Updated] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-2008: -- Attachment: YARN-2008.6.patch add small epsilon to assertions CapacityScheduler may report incorrect

[jira] [Updated] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-2008: -- Attachment: YARN-2008.7.patch and, this time with the change... CapacityScheduler may report

[jira] [Updated] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-2008: -- Attachment: YARN-2008.8.patch Make ResourceCalculator.isInvalidDivisor abstract, move (correct) impls

[jira] [Commented] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14088488#comment-14088488 ] Craig Welch commented on YARN-2008: --- TestAMRestart passes on my box with the changes,

[jira] [Updated] (YARN-2008) CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure

[ https://issues.apache.org/jira/browse/YARN-2008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-2008: -- Attachment: YARN-2008.9.patch Add specific tests for invalid divisor for both default and dominant

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093216#comment-14093216 ] Craig Welch commented on YARN-1198: --- So, I'm in the process of putting together a patch

[jira] [Updated] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1198: -- Attachment: YARN-1198.2.patch Patch for most items in this jira not covered in subtasks, including

[jira] [Updated] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1198: -- Attachment: YARN-1198.3.patch Update broken test to reflect intentional change in behavior Capacity

[jira] [Updated] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1198: -- Attachment: YARN-1198.4.patch redo-patch against current trunk, trigger rebuild b/c something went

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14094997#comment-14094997 ] Craig Welch commented on YARN-1198: --- So, looking at this a bit more holistically - it

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14097275#comment-14097275 ] Craig Welch commented on YARN-1198: --- So, it's possible to avoid iterating the

[jira] [Updated] (YARN-1857) CapacityScheduler headroom doesn't account for other AM's running

[ https://issues.apache.org/jira/browse/YARN-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1857: -- Attachment: YARN-1857.1.patch Just updating to a patch which applies against current trunk, otherwise

[jira] [Updated] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1198: -- Attachment: YARN-1198.5.patch Patch with a slightly different approach - calculates final headroom on

[jira] [Updated] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1198: -- Attachment: YARN-1198.6.patch Fix for findbug finds Capacity Scheduler headroom calculation does not

[jira] [Updated] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1198: -- Attachment: YARN-1198.7.patch Can't repro test failure - uploading (better formatted) patch to trigger

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109738#comment-14109738 ] Craig Welch commented on YARN-1198: --- I initially considered an approach like this one,

[jira] [Commented] (YARN-1857) CapacityScheduler headroom doesn't account for other AM's running

[ https://issues.apache.org/jira/browse/YARN-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109798#comment-14109798 ] Craig Welch commented on YARN-1857: --- [~jianhe] [~wangda] could you have a look at this

[jira] [Updated] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1198: -- Attachment: YARN-1198.8.patch Patch based on my last comment which iterates/calculates headroom at the

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14122188#comment-14122188 ] Craig Welch commented on YARN-1198: --- [~jianhe], have a look at patch 7, it takes that

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14127329#comment-14127329 ] Craig Welch commented on YARN-796: -- This is a bit of a detail, but the current version of

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129024#comment-14129024 ] Craig Welch commented on YARN-796: -- So, I'm adding code to check whether a user should be

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129480#comment-14129480 ] Craig Welch commented on YARN-796: -- Good, what you describe wrt the cli is what I was

[jira] [Updated] (YARN-1857) CapacityScheduler headroom doesn't account for other AM's running

[ https://issues.apache.org/jira/browse/YARN-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1857: -- Attachment: YARN-1857.2.patch I went ahead and put in a comment wrt a headroom value in the test and

[jira] [Commented] (YARN-2496) [YARN-796] Changes for capacity scheduler to support allocate resource respect labels

[ https://issues.apache.org/jira/browse/YARN-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135772#comment-14135772 ] Craig Welch commented on YARN-2496: --- LeafQueue.java I think there’s a bug in the way a

[jira] [Commented] (YARN-2494) [YARN-796] Node label manager API and storage implementations

[ https://issues.apache.org/jira/browse/YARN-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135774#comment-14135774 ] Craig Welch commented on YARN-2494: --- UpdateNodeResourceRequestPBImple.java: the only

[jira] [Updated] (YARN-1857) CapacityScheduler headroom doesn't account for other AM's running

[ https://issues.apache.org/jira/browse/YARN-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1857: -- Attachment: YARN-1857.3.patch re attaching patch to get the javac warnings CapacityScheduler headroom

[jira] [Commented] (YARN-2496) [YARN-796] Changes for capacity scheduler to support allocate resource respect labels

[ https://issues.apache.org/jira/browse/YARN-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141342#comment-14141342 ] Craig Welch commented on YARN-2496: --- Let's say user UserU is running two jobs, JobA and

[jira] [Commented] (YARN-2496) [YARN-796] Changes for capacity scheduler to support allocate resource respect labels

[ https://issues.apache.org/jira/browse/YARN-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141355#comment-14141355 ] Craig Welch commented on YARN-2496: --- I'm also concerned about this: Resource

[jira] [Commented] (YARN-2496) [YARN-796] Changes for capacity scheduler to support allocate resource respect labels

[ https://issues.apache.org/jira/browse/YARN-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141367#comment-14141367 ] Craig Welch commented on YARN-2496: --- On the parent/leaf refactor to share AbstractCSQueue

[jira] [Commented] (YARN-2496) [YARN-796] Changes for capacity scheduler to support allocate resource respect labels

[ https://issues.apache.org/jira/browse/YARN-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141369#comment-14141369 ] Craig Welch commented on YARN-2496: --- CSQueueUtils - just removing a line, should revert

[jira] [Commented] (YARN-2496) [YARN-796] Changes for capacity scheduler to support allocate resource respect labels

[ https://issues.apache.org/jira/browse/YARN-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141508#comment-14141508 ] Craig Welch commented on YARN-2496: --- SchedulerUtils.checkNodeLabelExpression - I think

[jira] [Commented] (YARN-2496) [YARN-796] Changes for capacity scheduler to support allocate resource respect labels

[ https://issues.apache.org/jira/browse/YARN-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141724#comment-14141724 ] Craig Welch commented on YARN-2496: --- Ok, so it sounds like 1, 3 and 4 are ok. I think

[jira] [Commented] (YARN-2496) [YARN-796] Changes for capacity scheduler to support allocate resource respect labels

[ https://issues.apache.org/jira/browse/YARN-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141736#comment-14141736 ] Craig Welch commented on YARN-2496: --- ach, not finished - anyway, re 2 - a particular job

[jira] [Commented] (YARN-2494) [YARN-796] Node label manager API and storage implementations

[ https://issues.apache.org/jira/browse/YARN-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14143665#comment-14143665 ] Craig Welch commented on YARN-2494: --- The other day [~vinodkv] suggested changing the

[jira] [Commented] (YARN-2505) [YARN-796] Support get/add/remove/change labels in RM REST API

[ https://issues.apache.org/jira/browse/YARN-2505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14143855#comment-14143855 ] Craig Welch commented on YARN-2505: --- 1) -re rename all-nodes-to-lables to

[jira] [Updated] (YARN-2505) [YARN-796] Support get/add/remove/change labels in RM REST API

[ https://issues.apache.org/jira/browse/YARN-2505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-2505: -- Attachment: YARN-2505.1.patch [YARN-796] Support get/add/remove/change labels in RM REST API

[jira] [Commented] (YARN-2496) [YARN-796] Changes for capacity scheduler to support allocate resource respect labels

[ https://issues.apache.org/jira/browse/YARN-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14143993#comment-14143993 ] Craig Welch commented on YARN-2496: --- So, re the headroom issue (2) - the short version -

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14144090#comment-14144090 ] Craig Welch commented on YARN-796: -- It looks like the FileSystemNodeLabelManager will just

[jira] [Commented] (YARN-2198) Remove the need to run NodeManager as privileged account for Windows Secure Container Executor

[ https://issues.apache.org/jira/browse/YARN-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14144437#comment-14144437 ] Craig Welch commented on YARN-2198: --- Looks good to me on the whole, a nit - :

[jira] [Commented] (YARN-796) Allow for (admin) labels on nodes and resource-requests

[ https://issues.apache.org/jira/browse/YARN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14147034#comment-14147034 ] Craig Welch commented on YARN-796: -- Some additional info regarding the headroom problem -

[jira] [Commented] (YARN-2494) [YARN-796] Node label manager API and storage implementations

[ https://issues.apache.org/jira/browse/YARN-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148094#comment-14148094 ] Craig Welch commented on YARN-2494: --- -re I suggest to change addLabels to addNodeLabels

[jira] [Commented] (YARN-2494) [YARN-796] Node label manager API and storage implementations

[ https://issues.apache.org/jira/browse/YARN-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148095#comment-14148095 ] Craig Welch commented on YARN-2494: --- ...other kinds of labels..., rather [YARN-796]

[jira] [Commented] (YARN-2494) [YARN-796] Node label manager API and storage implementations

[ https://issues.apache.org/jira/browse/YARN-2494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152078#comment-14152078 ] Craig Welch commented on YARN-2494: --- Not to dither about names - but Collection is still

[jira] [Commented] (YARN-1063) Winutils needs ability to create task as domain user

[ https://issues.apache.org/jira/browse/YARN-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152504#comment-14152504 ] Craig Welch commented on YARN-1063: --- When looking this over to pickup context for 2198, I

[jira] [Commented] (YARN-2198) Remove the need to run NodeManager as privileged account for Windows Secure Container Executor

[ https://issues.apache.org/jira/browse/YARN-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152523#comment-14152523 ] Craig Welch commented on YARN-2198: --- pom.xml - don’t see a /etc/hadoop or a

[jira] [Commented] (YARN-1972) Implement secure Windows Container Executor

[ https://issues.apache.org/jira/browse/YARN-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14153491#comment-14153491 ] Craig Welch commented on YARN-1972: --- [~rusanu] [~vinodkv], as on [YARN-1063], we can go

[jira] [Commented] (YARN-1063) Winutils needs ability to create task as domain user

[ https://issues.apache.org/jira/browse/YARN-1063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14153490#comment-14153490 ] Craig Welch commented on YARN-1063: --- [~rusanu] [~vinodkv], we can go ahead and address

[jira] [Commented] (YARN-2198) Remove the need to run NodeManager as privileged account for Windows Secure Container Executor

[ https://issues.apache.org/jira/browse/YARN-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14153494#comment-14153494 ] Craig Welch commented on YARN-2198: --- Bringing over some comments from [YARN-1063] When

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14153499#comment-14153499 ] Craig Welch commented on YARN-1198: --- That's not intentional - I think it's just a side

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14153512#comment-14153512 ] Craig Welch commented on YARN-1198: --- [~leftnoteasy] [~john.jian.fang] it sounds like the

[jira] [Commented] (YARN-2198) Remove the need to run NodeManager as privileged account for Windows Secure Container Executor

[ https://issues.apache.org/jira/browse/YARN-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14153982#comment-14153982 ] Craig Welch commented on YARN-2198: --- -re pom.xml - maybe I'm just confused, I saw a

[jira] [Commented] (YARN-2198) Remove the need to run NodeManager as privileged account for Windows Secure Container Executor

[ https://issues.apache.org/jira/browse/YARN-2198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14153983#comment-14153983 ] Craig Welch commented on YARN-2198: --- there are a number of changes which impact common

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14154057#comment-14154057 ] Craig Welch commented on YARN-1198: --- Sorry, the above was off - the conversation happened

[jira] [Commented] (YARN-1680) availableResources sent to applicationMaster in heartbeat should exclude blacklistedNodes free memory.

[ https://issues.apache.org/jira/browse/YARN-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14155295#comment-14155295 ] Craig Welch commented on YARN-1680: --- As I recall, blacklisted nodes are application level

[jira] [Commented] (YARN-1680) availableResources sent to applicationMaster in heartbeat should exclude blacklistedNodes free memory.

[ https://issues.apache.org/jira/browse/YARN-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14155360#comment-14155360 ] Craig Welch commented on YARN-1680: --- There are different kinds of blacklisting, the one

[jira] [Updated] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1198: -- Attachment: YARN-1198.9.patch Updated version of .7 patch to current trunk (as .7 now fails to fully

[jira] [Updated] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1198: -- Attachment: YARN-1198.10.patch And again, this time, with the additional files, .10 (nee .9, .7)

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14155767#comment-14155767 ] Craig Welch commented on YARN-1198: --- The Jenkins failures do not actually seem to have

[jira] [Commented] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14155764#comment-14155764 ] Craig Welch commented on YARN-1198: --- [~john.jian.fang] I look a look at implementing the

[jira] [Updated] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1198: -- Attachment: YARN-1198.11.patch Attaching patch .11, this is based on .10 (nee .7), the preferred

[jira] [Updated] (YARN-1198) Capacity Scheduler headroom calculation does not work as expected

[ https://issues.apache.org/jira/browse/YARN-1198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Craig Welch updated YARN-1198: -- Attachment: YARN-1198.11-with-1857.patch Patch combining the last .11 with the latest 1857 patch, to

[jira] [Commented] (YARN-1680) availableResources sent to applicationMaster in heartbeat should exclude blacklistedNodes free memory.

[ https://issues.apache.org/jira/browse/YARN-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14157248#comment-14157248 ] Craig Welch commented on YARN-1680: --- [~airbots] thanks for your updated WIP patch - I've

[jira] [Commented] (YARN-1680) availableResources sent to applicationMaster in heartbeat should exclude blacklistedNodes free memory.

[ https://issues.apache.org/jira/browse/YARN-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14157271#comment-14157271 ] Craig Welch commented on YARN-1680: --- [~john.jian.fang] I should probably not have

  1   2   3   4   5   >