[jira] [Updated] (YARN-1655) Add implementations to FairScheduler to support increase/decrease container resource

2019-08-01 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-1655: Attachment: YARN-1655.005.patch > Add implementations to FairScheduler to support

[jira] [Commented] (YARN-1655) Add implementations to FairScheduler to support increase/decrease container resource

2019-08-01 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898466#comment-16898466 ] Wilfred Spiegelenburg commented on YARN-1655: - the failed unit test is flaky as per YARN-8433

[jira] [Commented] (YARN-1655) Add implementations to FairScheduler to support increase/decrease container resource

2019-08-01 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897819#comment-16897819 ] Wilfred Spiegelenburg commented on YARN-1655: - Thank you for the feedback [~snemeth], sorry

[jira] [Updated] (YARN-1655) Add implementations to FairScheduler to support increase/decrease container resource

2019-08-01 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-1655: Attachment: YARN-1655.004.patch > Add implementations to FairScheduler to support

[jira] [Commented] (YARN-7621) Support submitting apps with queue path for CapacityScheduler

2019-07-31 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16897745#comment-16897745 ] Wilfred Spiegelenburg commented on YARN-7621: - You will need to support duplicate leaf queue

[jira] [Resolved] (YARN-9516) move application between queues,not check target queue acl permission

2019-05-01 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg resolved YARN-9516. - Resolution: Duplicate This has been fixed as YARN-5554

[jira] [Commented] (YARN-9431) Fix flaky junit test fair.TestAppRunnability after YARN-8967

2019-04-01 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807337#comment-16807337 ] Wilfred Spiegelenburg commented on YARN-9431: - Thank you [~giovanni.fumarola] for the commit

[jira] [Created] (YARN-9431) flaky junit test fair.TestAppRunnability after YARN-8967

2019-03-31 Thread Wilfred Spiegelenburg (JIRA)
Wilfred Spiegelenburg created YARN-9431: --- Summary: flaky junit test fair.TestAppRunnability after YARN-8967 Key: YARN-9431 URL: https://issues.apache.org/jira/browse/YARN-9431 Project: Hadoop

[jira] [Commented] (YARN-4901) MockRM should clear the QueueMetrics when it starts

2019-03-31 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806326#comment-16806326 ] Wilfred Spiegelenburg commented on YARN-4901: - I have run the test over 2500 times and cannot

[jira] [Created] (YARN-9417) Implement FS equivalent of AppNameMappingPlacementRule

2019-03-27 Thread Wilfred Spiegelenburg (JIRA)
Wilfred Spiegelenburg created YARN-9417: --- Summary: Implement FS equivalent of AppNameMappingPlacementRule Key: YARN-9417 URL: https://issues.apache.org/jira/browse/YARN-9417 Project: Hadoop YARN

[jira] [Comment Edited] (YARN-9416) Add filter options to FS placement rules

2019-03-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802761#comment-16802761 ] Wilfred Spiegelenburg edited comment on YARN-9416 at 3/27/19 1:22 PM: --

[jira] [Commented] (YARN-9416) Add filter options to FS placement rules

2019-03-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802761#comment-16802761 ] Wilfred Spiegelenburg commented on YARN-9416: - The proposal is to add a new child entry to all

[jira] [Created] (YARN-9416) Add filter options to FS placement rules

2019-03-27 Thread Wilfred Spiegelenburg (JIRA)
Wilfred Spiegelenburg created YARN-9416: --- Summary: Add filter options to FS placement rules Key: YARN-9416 URL: https://issues.apache.org/jira/browse/YARN-9416 Project: Hadoop YARN

[jira] [Commented] (YARN-8793) QueuePlacementPolicy bind more information to assigning result

2019-03-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802721#comment-16802721 ] Wilfred Spiegelenburg commented on YARN-8793: - The PlacementRule and PlacementManager have

[jira] [Resolved] (YARN-5387) FairScheduler: add the ability to specify a parent queue to all placement rules

2019-03-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg resolved YARN-5387. - Resolution: Implemented This has been included as part of the YARN-8967 changes.

[jira] [Commented] (YARN-8795) QueuePlacementRule move to separate files

2019-03-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802708#comment-16802708 ] Wilfred Spiegelenburg commented on YARN-8795: - The rules have been moved as part of the move

[jira] [Commented] (YARN-8792) Revisit FairScheduler QueuePlacementPolicy

2019-03-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802707#comment-16802707 ] Wilfred Spiegelenburg commented on YARN-8792: - None of these changes fit into the integrated

[jira] [Resolved] (YARN-2257) Add user to queue mappings to automatically place users' apps into specific queues

2019-03-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg resolved YARN-2257. - Resolution: Duplicate This has been fixed as part of YARN-8948, YARN-9298 and

[jira] [Created] (YARN-9415) Document FS placement rule changes from YARN-8967

2019-03-27 Thread Wilfred Spiegelenburg (JIRA)
Wilfred Spiegelenburg created YARN-9415: --- Summary: Document FS placement rule changes from YARN-8967 Key: YARN-9415 URL: https://issues.apache.org/jira/browse/YARN-9415 Project: Hadoop YARN

[jira] [Assigned] (YARN-6567) Flexible Workload Management

2019-03-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg reassigned YARN-6567: --- Assignee: Wilfred Spiegelenburg > Flexible Workload Management >

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-26 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801415#comment-16801415 ] Wilfred Spiegelenburg commented on YARN-8967: - Thank you [~yufeigu] for the commit. > Change

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-22 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16799020#comment-16799020 ] Wilfred Spiegelenburg commented on YARN-8967: - 3) I need two pieces back from the child when

[jira] [Updated] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-22 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-8967: Attachment: YARN-8967.011.patch > Change FairScheduler to use PlacementRule

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-21 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16798645#comment-16798645 ] Wilfred Spiegelenburg commented on YARN-8967: - Junit test failure is not related. The

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-21 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797873#comment-16797873 ] Wilfred Spiegelenburg commented on YARN-8967: - 1) I missed that one too, fixed now 3) The two

[jira] [Updated] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-21 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-8967: Attachment: YARN-8967.010.patch > Change FairScheduler to use PlacementRule

[jira] [Commented] (YARN-9278) Shuffle nodes when selecting to be preempted nodes

2019-03-19 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16796823#comment-16796823 ] Wilfred Spiegelenburg commented on YARN-9278: - Thank for the update [~uranus] I don't think

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-18 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795640#comment-16795640 ] Wilfred Spiegelenburg commented on YARN-8967: - Cleaned up the checkstyle issues and fixed the

[jira] [Updated] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-18 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-8967: Attachment: YARN-8967.009.patch > Change FairScheduler to use PlacementRule

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-18 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795049#comment-16795049 ] Wilfred Spiegelenburg commented on YARN-8967: - Thank you for the review [~yufeigu] 1) yes it

[jira] [Comment Edited] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-18 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795049#comment-16795049 ] Wilfred Spiegelenburg edited comment on YARN-8967 at 3/18/19 2:15 PM: --

[jira] [Updated] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-18 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-8967: Attachment: YARN-8967.008.patch > Change FairScheduler to use PlacementRule

[jira] [Commented] (YARN-9278) Shuffle nodes when selecting to be preempted nodes

2019-03-17 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16794653#comment-16794653 ] Wilfred Spiegelenburg commented on YARN-9278: - Thank you for the update. [~uranus] The code

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-12 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791169#comment-16791169 ] Wilfred Spiegelenburg commented on YARN-8967: - [~haibochen] or [~templedf] could either of you

[jira] [Commented] (YARN-9278) Shuffle nodes when selecting to be preempted nodes

2019-03-12 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790501#comment-16790501 ] Wilfred Spiegelenburg commented on YARN-9278: - I have a couple of comments on the patch. * I

[jira] [Commented] (YARN-9344) FS should not reserve when container capability is bigger than node total resource

2019-03-12 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790510#comment-16790510 ] Wilfred Spiegelenburg commented on YARN-9344: - Thank you [~uranus] for adding the test. Could

[jira] [Comment Edited] (YARN-9278) Shuffle nodes when selecting to be preempted nodes

2019-03-12 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16780101#comment-16780101 ] Wilfred Spiegelenburg edited comment on YARN-9278 at 3/12/19 12:22 PM:

[jira] [Comment Edited] (YARN-9278) Shuffle nodes when selecting to be preempted nodes

2019-03-12 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16780101#comment-16780101 ] Wilfred Spiegelenburg edited comment on YARN-9278 at 3/12/19 12:25 PM:

[jira] [Commented] (YARN-9314) Fair Scheduler: Queue Info mistake when configured same queue name at same level

2019-03-07 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786734#comment-16786734 ] Wilfred Spiegelenburg commented on YARN-9314: - Hi [~fengyongshe], thank you for filing this

[jira] [Commented] (YARN-9343) Replace isDebugEnabled with SLF4J parameterized log messages

2019-03-07 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786572#comment-16786572 ] Wilfred Spiegelenburg commented on YARN-9343: - yes I am fine with that. This patch is big

[jira] [Comment Edited] (YARN-9343) Replace isDebugEnabled with SLF4J parameterized log messages

2019-03-06 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786336#comment-16786336 ] Wilfred Spiegelenburg edited comment on YARN-9343 at 3/7/19 3:28 AM: -

[jira] [Commented] (YARN-9343) Replace isDebugEnabled with SLF4J parameterized log messages

2019-03-06 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786336#comment-16786336 ] Wilfred Spiegelenburg commented on YARN-9343: - Thank you for the update [~Prabhu Joseph] I

[jira] [Commented] (YARN-9344) FS should not reserve when container capability is bigger than node total resource

2019-03-06 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786314#comment-16786314 ] Wilfred Spiegelenburg commented on YARN-9344: - The test failures are also not related:

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-06 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786306#comment-16786306 ] Wilfred Spiegelenburg commented on YARN-8967: - Fixed the newly introduced checkstyle issues.

[jira] [Updated] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-06 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-8967: Attachment: YARN-8967.007.patch > Change FairScheduler to use PlacementRule

[jira] [Commented] (YARN-9326) Fair Scheduler configuration defaults are not documented in case of min and maxResources

2019-03-06 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16786290#comment-16786290 ] Wilfred Spiegelenburg commented on YARN-9326: - The white space issues are fixed via YARN-9348.

[jira] [Updated] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-06 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-8967: Attachment: (was: YARN-8967.006.patch) > Change FairScheduler to use

[jira] [Updated] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-03-06 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-8967: Attachment: YARN-8967.006.patch > Change FairScheduler to use PlacementRule

[jira] [Commented] (YARN-9343) Replace isDebugEnabled with SLF4J parameterized log messages

2019-03-05 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785182#comment-16785182 ] Wilfred Spiegelenburg commented on YARN-9343: - Hi [~Prabhu Joseph] I see a lot of changes

[jira] [Commented] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-03-05 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785164#comment-16785164 ] Wilfred Spiegelenburg commented on YARN-9298: - Thank you [~yufeigu] I will follow up with the

[jira] [Commented] (YARN-9341) Reentrant lock() before try

2019-03-05 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785162#comment-16785162 ] Wilfred Spiegelenburg commented on YARN-9341: - So we do have one and also one

[jira] [Commented] (YARN-9344) FS should not reserve when container capability is bigger than node total resource

2019-03-05 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16785103#comment-16785103 ] Wilfred Spiegelenburg commented on YARN-9344: - HI [~uranus] yes this is a problem nice catch.

[jira] [Commented] (YARN-9341) Reentrant lock() before try

2019-03-04 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783987#comment-16783987 ] Wilfred Spiegelenburg commented on YARN-9341: - An {{IllegalMonitorStateException}} can only

[jira] [Commented] (YARN-9343) Replace isDebugEnabled with SLF4J parameterized log messages

2019-03-04 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783952#comment-16783952 ] Wilfred Spiegelenburg commented on YARN-9343: - Hi [~Prabhu Joseph], thank you for this update.

[jira] [Commented] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-03-03 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782932#comment-16782932 ] Wilfred Spiegelenburg commented on YARN-9298: - # saw that, had them already fixed in a new

[jira] [Updated] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-03-03 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-9298: Attachment: YARN-9298.006.patch > Implement FS placement rules using PlacementRule

[jira] [Commented] (YARN-6487) FairScheduler: remove continuous scheduling (YARN-1010)

2019-03-03 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16782897#comment-16782897 ] Wilfred Spiegelenburg commented on YARN-6487: - It works both ways: Before we can schedule we

[jira] [Commented] (YARN-9326) Fair Scheduler configuration defaults are not documented in case of min and maxResources

2019-03-01 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781720#comment-16781720 ] Wilfred Spiegelenburg commented on YARN-9326: - Thank you for the update [~adam.antal] I see

[jira] [Commented] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-03-01 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781661#comment-16781661 ] Wilfred Spiegelenburg commented on YARN-9298: - Last version also fixes the {{createQueue}}

[jira] [Updated] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-03-01 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-9298: Attachment: YARN-9298.005.patch > Implement FS placement rules using PlacementRule

[jira] [Commented] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-03-01 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781648#comment-16781648 ] Wilfred Spiegelenburg commented on YARN-9298: - # I added the abstract {{FSPlacementRule}} and

[jira] [Updated] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-03-01 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-9298: Attachment: YARN-9298.004.patch > Implement FS placement rules using PlacementRule

[jira] [Commented] (YARN-6487) FairScheduler: remove continuous scheduling (YARN-1010)

2019-02-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16780108#comment-16780108 ] Wilfred Spiegelenburg commented on YARN-6487: - The removal of continuous scheduling was/is

[jira] [Commented] (YARN-9278) Shuffle nodes when selecting to be preempted nodes

2019-02-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16780101#comment-16780101 ] Wilfred Spiegelenburg commented on YARN-9278: - Two things: * I still think limiting the number

[jira] [Comment Edited] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-02-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779358#comment-16779358 ] Wilfred Spiegelenburg edited comment on YARN-9298 at 2/28/19 2:11 AM: --

[jira] [Commented] (YARN-9326) Fair Scheduler configuration defaults are not documented in case of min and maxResources

2019-02-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16780015#comment-16780015 ] Wilfred Spiegelenburg commented on YARN-9326: - Thank you for the updated patch [~adam.antal]

[jira] [Updated] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-02-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-8967: Attachment: YARN-8967.006.patch > Change FairScheduler to use PlacementRule

[jira] [Updated] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-02-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-9298: Attachment: YARN-9298.003.patch > Implement FS placement rules using PlacementRule

[jira] [Commented] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-02-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779358#comment-16779358 ] Wilfred Spiegelenburg commented on YARN-9298: - 1) oops copy past error, fixed now 2) yep,

[jira] [Commented] (YARN-9326) Fair Scheduler configuration defaults are not documented in case of min and maxResources

2019-02-27 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779296#comment-16779296 ] Wilfred Spiegelenburg commented on YARN-9326: - Thanks [~adam.antal] for following up on

[jira] [Commented] (YARN-9278) Shuffle nodes when selecting to be preempted nodes

2019-02-24 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776502#comment-16776502 ] Wilfred Spiegelenburg commented on YARN-9278: - [~uranus] I can understand that you want to

[jira] [Commented] (YARN-9323) FSLeafQueue#computeMaxAMResource does not override zero values for custom resources

2019-02-21 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774779#comment-16774779 ] Wilfred Spiegelenburg commented on YARN-9323: - Hi [~snemeth] Some comments on this change as

[jira] [Commented] (YARN-8662) Fair Scheduler stops scheduling when a queue is configured only CPU and memory

2019-02-21 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774708#comment-16774708 ] Wilfred Spiegelenburg commented on YARN-8662: - Hi [~Sen Zhao], Thank you for filing this and

[jira] [Commented] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-02-20 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773708#comment-16773708 ] Wilfred Spiegelenburg commented on YARN-9298: - Thank you for the review [~yufeigu] it took a

[jira] [Updated] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-02-20 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-9298: Attachment: YARN-9298.002.patch > Implement FS placement rules using PlacementRule

[jira] [Commented] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-02-14 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16768900#comment-16768900 ] Wilfred Spiegelenburg commented on YARN-9298: - [~cheersyang] Can you please check this? >

[jira] [Updated] (YARN-9308) fairscheduler-statedump.log gets generated regardless of service again after the merge of HDFS-7240

2019-02-14 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-9308: Attachment: YARN-9308.001.patch > fairscheduler-statedump.log gets generated

[jira] [Commented] (YARN-9308) fairscheduler-statedump.log gets generated regardless of service again after the merge of HDFS-7240

2019-02-14 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16768891#comment-16768891 ] Wilfred Spiegelenburg commented on YARN-9308: - The changes from [HDFS-7240 git commit

[jira] [Assigned] (YARN-9308) fairscheduler-statedump.log gets generated regardless of service again after the merge of HDFS-7240

2019-02-14 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg reassigned YARN-9308: --- Assignee: Wilfred Spiegelenburg > fairscheduler-statedump.log gets generated

[jira] [Commented] (YARN-1655) Add implementations to FairScheduler to support increase/decrease container resource

2019-02-13 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767164#comment-16767164 ] Wilfred Spiegelenburg commented on YARN-1655: - The junit test failures are not related to this

[jira] [Commented] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-02-13 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767167#comment-16767167 ] Wilfred Spiegelenburg commented on YARN-9298: - Junit test failure seems unrelated no tests is

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-02-12 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16766813#comment-16766813 ] Wilfred Spiegelenburg commented on YARN-8967: - After talking off line with a number of people

[jira] [Commented] (YARN-9277) Add more restrictions In FairScheduler Preemption

2019-02-12 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16766811#comment-16766811 ] Wilfred Spiegelenburg commented on YARN-9277: - I agree with [~Steven Rand] sorting could be

[jira] [Updated] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-02-12 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-9298: Attachment: YARN-9298.001.patch > Implement FS placement rules using PlacementRule

[jira] [Created] (YARN-9298) Implement FS placement rules using PlacementRule interface

2019-02-12 Thread Wilfred Spiegelenburg (JIRA)
Wilfred Spiegelenburg created YARN-9298: --- Summary: Implement FS placement rules using PlacementRule interface Key: YARN-9298 URL: https://issues.apache.org/jira/browse/YARN-9298 Project: Hadoop

[jira] [Commented] (YARN-1655) Add implementations to FairScheduler to support increase/decrease container resource

2019-02-12 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16766771#comment-16766771 ] Wilfred Spiegelenburg commented on YARN-1655: - Updated test to make it more robust. locally

[jira] [Updated] (YARN-1655) Add implementations to FairScheduler to support increase/decrease container resource

2019-02-12 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-1655: Attachment: YARN-1655.003.patch > Add implementations to FairScheduler to support

[jira] [Commented] (YARN-8655) FairScheduler: FSStarvedApps is not thread safe

2019-02-12 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765949#comment-16765949 ] Wilfred Spiegelenburg commented on YARN-8655: - Hi [~uranus], I am not saying that what we do

[jira] [Commented] (YARN-8655) FairScheduler: FSStarvedApps is not thread safe

2019-02-11 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765733#comment-16765733 ] Wilfred Spiegelenburg commented on YARN-8655: - Looking at how we get to adding an application

[jira] [Commented] (YARN-1655) Add implementations to FairScheduler to support increase/decrease container resource

2019-02-11 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765663#comment-16765663 ] Wilfred Spiegelenburg commented on YARN-1655: -

[jira] [Updated] (YARN-1655) Add implementations to FairScheduler to support increase/decrease container resource

2019-02-11 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-1655: Attachment: YARN-1655.002.patch > Add implementations to FairScheduler to support

[jira] [Updated] (YARN-1655) Add implementations to FairScheduler to support increase/decrease container resource

2019-02-11 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-1655: Attachment: YARN-1655.001.patch > Add implementations to FairScheduler to support

[jira] [Comment Edited] (YARN-1655) Add implementations to FairScheduler to support increase/decrease container resource

2019-02-11 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16764807#comment-16764807 ] Wilfred Spiegelenburg edited comment on YARN-1655 at 2/11/19 9:50 AM: --

[jira] [Updated] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-02-03 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-8967: Attachment: YARN-8967.005.patch > Change FairScheduler to use PlacementRule

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-02-03 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759579#comment-16759579 ] Wilfred Spiegelenburg commented on YARN-8967: - Rebased to trunk, the mockito changes prevented

[jira] [Commented] (YARN-9262) TestRMAppAttemptTransitions is failing with an NPE

2019-02-01 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758145#comment-16758145 ] Wilfred Spiegelenburg commented on YARN-9262: - The failure occurs because no {{Allocation}}

[jira] [Commented] (YARN-1655) Add implementations to FairScheduler to support increase/decrease container resource

2019-02-01 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758120#comment-16758120 ] Wilfred Spiegelenburg commented on YARN-1655: - I have started working on this already. I have

[jira] [Assigned] (YARN-1655) Add implementations to FairScheduler to support increase/decrease container resource

2019-02-01 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg reassigned YARN-1655: --- Assignee: Wilfred Spiegelenburg (was: Sandy Ryza) > Add implementations to

[jira] [Commented] (YARN-8967) Change FairScheduler to use PlacementRule interface

2019-01-31 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757815#comment-16757815 ] Wilfred Spiegelenburg commented on YARN-8967: - [~sunilg] can you please have a review of this

[jira] [Commented] (YARN-9198) Corrupted state from a previous version can still cause RM to fail with NPE on FairScheduler

2019-01-15 Thread Wilfred Spiegelenburg (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743604#comment-16743604 ] Wilfred Spiegelenburg commented on YARN-9198: - Not restoring an application is irreversible.

<    1   2   3   4   5   6   >