[jira] [Updated] (YARN-3828) Add a flag in container to indicate whether it's an AM container or not

2015-07-28 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3828: -- Assignee: Giovanni Matteo Fumarola (was: Inigo Goiri) Add a flag in container to indicate whether it's

[jira] [Updated] (YARN-3534) Collect memory/cpu usage on the node

2015-07-31 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3534: -- Attachment: YARN-3534-16.patch Solved most of [~kasha]'s comments. Collect memory/cpu usage on the

[jira] [Commented] (YARN-3534) Collect memory/cpu usage on the node

2015-07-31 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14649993#comment-14649993 ] Inigo Goiri commented on YARN-3534: --- I guess in #2 you meant yarn-default.xml, right? I

[jira] [Updated] (YARN-3458) CPU resource monitoring in Windows

2015-08-14 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3458: -- Attachment: YARN-3458-7.patch Merging to trunk with the newest resource monitoring structure CPU

[jira] [Commented] (YARN-3534) Collect memory/cpu usage on the node

2015-08-15 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698419#comment-14698419 ] Inigo Goiri commented on YARN-3534: --- I uploaded 17 twice instead of 18... sorry.

[jira] [Updated] (YARN-3534) Collect memory/cpu usage on the node

2015-08-15 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3534: -- Attachment: YARN-3534-18.patch Fixed xml defaults. Collect memory/cpu usage on the node

[jira] [Commented] (YARN-313) Add Admin API for supporting node resource configuration in command line

2015-08-11 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14692632#comment-14692632 ] Inigo Goiri commented on YARN-313: -- Not critical, I think it can be deferred. I would

[jira] [Updated] (YARN-313) Add Admin API for supporting node resource configuration in command line

2015-08-11 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-313: - Attachment: YARN-313-v7.patch Updated to trunk. It looks like it still breaks the unit test for the

[jira] [Updated] (YARN-3458) CPU resource monitoring in Windows

2015-08-25 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3458: -- Attachment: YARN-3458-8.patch Rebase CPU resource monitoring in Windows

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: YARN-3980-v6.patch Fixed javadoc style issue. The checkstyle errors look like pedantic mode

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-13 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15004692#comment-15004692 ] Inigo Goiri commented on YARN-3980: --- Give us a couple hours. If it's not done by today we should go ahead

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-13 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005097#comment-15005097 ] Inigo Goiri commented on YARN-3980: --- We are failing miserably with the mini cluster. Do you guys mind

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-04 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: YARN-3980-v4.patch Adding utilizaiton to FIFO and Fair scheduler. > Plumb

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-04 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990284#comment-14990284 ] Inigo Goiri commented on YARN-3980: --- I added the info to the FIFO and Fair schedulers. I'm rerunning the

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-06 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994811#comment-14994811 ] Inigo Goiri commented on YARN-3980: --- OK, I'll try it for both. > Plumb resource-utilization info in node

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-06 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994805#comment-14994805 ] Inigo Goiri commented on YARN-3980: --- How does the violate thing work? Do you have any example? I'll

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-07 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: YARN-3980-v5.patch Moved ContainersUitlization to AggregatedContainersUtilization. Made

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-07 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: (was: YARN-3980-v5.patch) > Plumb resource-utilization info in node heartbeat through to

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-07 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14995372#comment-14995372 ] Inigo Goiri commented on YARN-3980: --- I uploaded a patch with all the changes but I'm not completely sure

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-07 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: YARN-3980-v5.patch Moved ContainersUitlization to AggregatedContainersUtilization. Made

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-07 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: (was: YARN-3980-v5.patch) > Plumb resource-utilization info in node heartbeat through to

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-07 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: YARN-3980-v5.patch Moved ContainersUitlization to AggregatedContainersUtilization. Made

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-10-12 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: YARN-3980-v3.patch Merging to trunk > Plumb resource-utilization info in node heartbeat

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-10-12 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: (was: YARN-3980-v3.patch) > Plumb resource-utilization info in node heartbeat through to

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-10-12 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: YARN-3980-v3.patch Missing key > Plumb resource-utilization info in node heartbeat through

[jira] [Updated] (YARN-313) Add Admin API for supporting node resource configuration in command line

2015-07-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-313: - Attachment: YARN-313-v6.patch Trying to fix the broken unit test for refreshNodes (I still don't understand

[jira] [Updated] (YARN-1012) Report NM aggregated container resource utilization in heartbeat

2015-07-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-1012: -- Attachment: YARN-1012-11.patch Fixed checkstyle issues (I hope the package-info is done properly). I

[jira] [Updated] (YARN-3534) Collect memory/cpu usage on the node

2015-07-09 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3534: -- Attachment: YARN-3534-14.patch Updated to trunk (using ResourceUtilization already there). Collect

[jira] [Commented] (YARN-1012) Report NM aggregated container resource utilization in heartbeat

2015-07-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14619829#comment-14619829 ] Inigo Goiri commented on YARN-1012: --- Thank you [~kasha]! Once commited, I'm moving to

[jira] [Updated] (YARN-3534) Collect memory/cpu usage on the node

2015-07-09 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3534: -- Attachment: YARN-3534-15.patch Fixed issue with constants. We need to agree that these names are OK and

[jira] [Commented] (YARN-1012) Report NM aggregated container resource utilization in heartbeat

2015-08-27 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14716178#comment-14716178 ] Inigo Goiri commented on YARN-1012: --- I think this is very YARN specific. It relies on the

[jira] [Updated] (YARN-313) Add Admin API for supporting node resource configuration in command line

2015-09-11 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-313: - Attachment: YARN-313-v10.patch Tackling [~djp] comments. > Add Admin API for supporting node resource

[jira] [Commented] (YARN-313) Add Admin API for supporting node resource configuration in command line

2015-09-11 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14740323#comment-14740323 ] Inigo Goiri commented on YARN-313: -- The patch I'm going to submit more or less fixes all of them, some

[jira] [Updated] (YARN-313) Add Admin API for supporting node resource configuration in command line

2015-09-11 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-313: - Attachment: YARN-313-v11.patch RefreshResource -> RefreshNodesResources > Add Admin API for supporting

[jira] [Updated] (YARN-313) Add Admin API for supporting node resource configuration in command line

2015-09-11 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-313: - Attachment: (was: YARN-313-v11.patch) > Add Admin API for supporting node resource configuration in

[jira] [Updated] (YARN-313) Add Admin API for supporting node resource configuration in command line

2015-09-11 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-313: - Attachment: YARN-313-v11.patch I forgot to add the RefreshNodesResources* files. > Add Admin API for

[jira] [Updated] (YARN-313) Add Admin API for supporting node resource configuration in command line

2015-09-09 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-313: - Attachment: YARN-313-v9.patch Rebase to trunk. > Add Admin API for supporting node resource configuration

[jira] [Updated] (YARN-3458) CPU resource monitoring in Windows

2015-12-02 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3458: -- Attachment: YARN-3458-9.patch Rebased to trunk. > CPU resource monitoring in Windows >

[jira] [Assigned] (YARN-2402) NM restart: Container recovery for Windows

2016-01-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri reassigned YARN-2402: - Assignee: Inigo Goiri > NM restart: Container recovery for Windows >

[jira] [Commented] (YARN-2402) NM restart: Container recovery for Windows

2016-01-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15089833#comment-15089833 ] Inigo Goiri commented on YARN-2402: --- I temporarily assigned the patch to me to trigger Jenkins. I tried

[jira] [Updated] (YARN-2882) Add ExecutionType to denote if a container execution is GUARANTEED or OPPORTUNISTIC

2015-12-26 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-2882: -- Attachment: YARN-2882.005.patch Missing ExecutionType. > Add ExecutionType to denote if a container

[jira] [Updated] (YARN-2882) Add ExecutionType to denote if a container execution is GUARANTEED or OPPORTUNISTIC

2015-12-26 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-2882: -- Attachment: (was: YARN-2882.005.patch) > Add ExecutionType to denote if a container execution is

[jira] [Updated] (YARN-2882) Add ExecutionType to denote if a container execution is GUARANTEED or OPPORTUNISTIC

2015-12-26 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-2882: -- Attachment: YARN-2882-v5.patch Tackling [~kasha] comments except rollingUpgrade. > Add ExecutionType to

[jira] [Updated] (YARN-2882) Add ExecutionType to denote if a container execution is GUARANTEED or OPPORTUNISTIC

2015-12-26 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-2882: -- Attachment: YARN-2882.005.patch Fixing patch. > Add ExecutionType to denote if a container execution is

[jira] [Updated] (YARN-2882) Add ExecutionType to denote if a container execution is GUARANTEED or OPPORTUNISTIC

2015-12-26 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-2882: -- Attachment: (was: YARN-2882-v5.patch) > Add ExecutionType to denote if a container execution is

[jira] [Commented] (YARN-1013) CS should watch resource utilization of containers and allocate speculative containers if appropriate

2015-12-27 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072104#comment-15072104 ] Inigo Goiri commented on YARN-1013: --- I can take this one once YARN-1015 is done. > CS should watch

[jira] [Commented] (YARN-1011) [Umbrella] Schedule containers based on utilization of currently allocated containers

2015-12-27 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072107#comment-15072107 ] Inigo Goiri commented on YARN-1011: --- The doc looks good. I have a couple questions: # What would be the

[jira] [Commented] (YARN-4512) Provide a knob to turn on over-subscription

2015-12-27 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072245#comment-15072245 ] Inigo Goiri commented on YARN-4512: --- Per the discussion in YARN-1011, it makes sense to add an option in

[jira] [Commented] (YARN-4330) MiniYARNCluster prints multiple Failed to instantiate default resource calculator warning messages

2015-12-27 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072148#comment-15072148 ] Inigo Goiri commented on YARN-4330: --- [~ste...@apache.org], is this good to go? > MiniYARNCluster prints

[jira] [Commented] (YARN-3458) CPU resource monitoring in Windows

2015-12-22 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15067745#comment-15067745 ] Inigo Goiri commented on YARN-3458: --- [~cnauroth], thank you very much for the review! > CPU resource

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-24 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025583#comment-15025583 ] Inigo Goiri commented on YARN-3980: --- Thank you for taking care of this! > Plumb resource-utilization

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-18 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15012423#comment-15012423 ] Inigo Goiri commented on YARN-3980: --- I think this unit tests cover what we wanted to test. Let's wait for

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-18 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15012897#comment-15012897 ] Inigo Goiri commented on YARN-3980: --- As with the previous one, the checkstyles seem a little bit pedantic

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-22 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021397#comment-15021397 ] Inigo Goiri commented on YARN-3980: --- It looks like v9 solves the issues. Let me know if you see anything

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-21 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15020804#comment-15020804 ] Inigo Goiri commented on YARN-3980: --- I'll take care of those changes. The only one we need to skip is #2

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-22 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: YARN-3980-v8.patch Tackling Karthik's comments. > Plumb resource-utilization info in node

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-22 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: (was: YARN-3980-v8.patch) > Plumb resource-utilization info in node heartbeat through to

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-22 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: YARN-3980-v8.patch Increasing timeout to 2 seconds again because unit tests fail. > Plumb

[jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-22 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021201#comment-15021201 ] Inigo Goiri commented on YARN-3980: --- Any easy place to check in the RM for the heartbeats? > Plumb

[jira] [Updated] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler

2015-11-22 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3980: -- Attachment: YARN-3980-v9.patch Waiting for heartbeat to propagate based on the expected value. This

[jira] [Commented] (YARN-4512) Provide a knob to turn on over-allocation

2016-01-12 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094493#comment-15094493 ] Inigo Goiri commented on YARN-4512: --- 003 looks good. I would like [~leftnoteasy] to take a look but I'd

[jira] [Commented] (YARN-4512) Provide a knob to turn on over-allocation

2016-01-10 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15091330#comment-15091330 ] Inigo Goiri commented on YARN-4512: --- Regarding the overallocation vs oversubscription, I'm more concerned

[jira] [Updated] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-03 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5171: -- Attachment: YARN-5171.004.patch Tackling [~asuresh] and [~kkaranasos] comments plus some style checks.

[jira] [Updated] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-03 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5171: -- Attachment: YARN-5171.005.patch Adding finish event when completing opportunistic containers. > Extend

[jira] [Created] (YARN-5215) Scheduling containers based on load in the servers

2016-06-08 Thread Inigo Goiri (JIRA)
Inigo Goiri created YARN-5215: - Summary: Scheduling containers based on load in the servers Key: YARN-5215 URL: https://issues.apache.org/jira/browse/YARN-5215 Project: Hadoop YARN Issue Type:

[jira] [Updated] (YARN-5215) Scheduling containers based on load in the servers

2016-06-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5215: -- Attachment: YARN-5215.000.patch First version to show the idea. > Scheduling containers based on load

[jira] [Updated] (YARN-5215) Scheduling containers based on external load in the servers

2016-06-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5215: -- Description: Currently YARN runs containers in the servers assuming that they own all the resources. The

[jira] [Updated] (YARN-5215) Scheduling containers based on external load in the servers

2016-06-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5215: -- Summary: Scheduling containers based on external load in the servers (was: Scheduling containers based

[jira] [Commented] (YARN-5215) Scheduling containers based on external load in the servers

2016-06-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321672#comment-15321672 ] Inigo Goiri commented on YARN-5215: --- Yes, I realized that the original title didn't mention external

[jira] [Commented] (YARN-5215) Scheduling containers based on external load in the servers

2016-06-09 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322829#comment-15322829 ] Inigo Goiri commented on YARN-5215: --- [~jlowe], my view on this feature is more like making the scheduler

[jira] [Comment Edited] (YARN-5215) Scheduling containers based on external load in the servers

2016-06-09 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322829#comment-15322829 ] Inigo Goiri edited comment on YARN-5215 at 6/9/16 4:38 PM: --- [~jlowe], my view on

[jira] [Commented] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-03 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15314962#comment-15314962 ] Inigo Goiri commented on YARN-5171: --- Regarding the checkstyle, the only one that doesn't break the rest

[jira] [Commented] (YARN-5215) Scheduling containers based on external load in the servers

2016-06-15 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332021#comment-15332021 ] Inigo Goiri commented on YARN-5215: --- [~ccurino], converting this to an umbrella makes sense to me.

[jira] [Commented] (YARN-5215) Scheduling containers based on external load in the servers

2016-06-13 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328631#comment-15328631 ] Inigo Goiri commented on YARN-5215: --- [~sunilg], I think your first two points are related. Let me try to

[jira] [Commented] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-06 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317528#comment-15317528 ] Inigo Goiri commented on YARN-5171: --- Reuse the test for distributed scheduling. > Extend

[jira] [Commented] (YARN-5215) Scheduling containers based on load in the servers

2016-06-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321355#comment-15321355 ] Inigo Goiri commented on YARN-5215: --- This could be a subtask in YARN-1011. However, I thought it was

[jira] [Commented] (YARN-5202) Dynamic Overcommit of Node Resources - POC

2016-06-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321810#comment-15321810 ] Inigo Goiri commented on YARN-5202: --- As mentioned in YARN-5215, I think this works fits pretty nicely

[jira] [Updated] (YARN-5215) Scheduling containers based on external load in the servers

2016-06-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5215: -- Attachment: YARN-5215.001.patch Adding configuration switch, boundary checks and unit tests. >

[jira] [Commented] (YARN-5215) Scheduling containers based on external load in the servers

2016-06-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321799#comment-15321799 ] Inigo Goiri commented on YARN-5215: --- Still fixing the unit test. > Scheduling containers based on

[jira] [Commented] (YARN-5215) Scheduling containers based on external load in the servers

2016-06-08 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321809#comment-15321809 ] Inigo Goiri commented on YARN-5215: --- [~curino], in our cluster we actually surface to the Web UI the

[jira] [Updated] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-02 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5171: -- Attachment: YARN-5171.003.patch Fixing compilation issues and style. > Extend

[jira] [Updated] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-02 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5171: -- Attachment: (was: YARN-5171.003.patch) > Extend DistributedSchedulerProtocol to notify RM of

[jira] [Commented] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-01 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311486#comment-15311486 ] Inigo Goiri commented on YARN-5171: --- [~kasha], I added the {{ExecutionType}} to {{RMContainer}} as it's

[jira] [Updated] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-01 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5171: -- Attachment: YARN-5171.001.patch Gettin {{ExecutionType}} from {{Container}}. > Extend

[jira] [Updated] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-01 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5171: -- Attachment: YARN-5171.002.patch Tackling [~asuresh] comments and check style. > Extend

[jira] [Updated] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-01 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5171: -- Attachment: YARN-5171.000.patch First proposal. > Extend DistributedSchedulerProtocol to notify RM of

[jira] [Commented] (YARN-5215) Scheduling containers based on external load in the servers

2016-06-14 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330649#comment-15330649 ] Inigo Goiri commented on YARN-5215: --- [~kasha], in our use case we are targeting co-locating with latency

[jira] [Updated] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-22 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5171: -- Attachment: YARN-5171.011.patch Fixing checktyles and naming. > Extend DistributedSchedulerProtocol to

[jira] [Updated] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-21 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5171: -- Attachment: YARN-5171.010.patch Adding RM check in the unit test. > Extend DistributedSchedulerProtocol

[jira] [Commented] (YARN-5215) Scheduling containers based on external load in the servers

2016-06-14 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330759#comment-15330759 ] Inigo Goiri commented on YARN-5215: --- My initial proposal was to add a generic support for external

[jira] [Updated] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-21 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5171: -- Attachment: YARN-5171.009.patch Trying to extend the unit test with opportunistic checks. > Extend

[jira] [Updated] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-19 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5171: -- Attachment: YARN-5171.007.patch Adding external to methods and changing {{ResourceUsage}}. Missing unit

[jira] [Commented] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-22 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345539#comment-15345539 ] Inigo Goiri commented on YARN-5171: --- [~asuresh], other than the unused import, everything else (including

[jira] [Updated] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-22 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5171: -- Attachment: YARN-5171.012.patch Added more comments and improved resiliency of unit test. > Extend

[jira] [Commented] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-20 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340950#comment-15340950 ] Inigo Goiri commented on YARN-5171: --- [~subru], agreed on the naming thing; external can be confusing with

[jira] [Updated] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-20 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5171: -- Attachment: YARN-5171.008.patch Adding one simple unit test and tweaking the names again. > Extend

[jira] [Commented] (YARN-5215) Scheduling containers based on external load in the servers

2016-06-15 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332483#comment-15332483 ] Inigo Goiri commented on YARN-5215: --- [~kkaranasos] I kind of like the "fake container" approach. In

[jira] [Updated] (YARN-5171) Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node

2016-06-16 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-5171: -- Attachment: YARN-5171.006.patch Tackling some of the comments from [~kkaranasos]. > Extend

[jira] [Commented] (YARN-4512) Provide a knob to turn on over-allocation

2016-01-10 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15091190#comment-15091190 ] Inigo Goiri commented on YARN-4512: --- A couple comments on the patch: # Some places (e.g.,

[jira] [Commented] (YARN-4512) Provide a knob to turn on over-allocation

2016-01-13 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15096891#comment-15096891 ] Inigo Goiri commented on YARN-4512: --- I think you both guys are saying the same: (1) the NM will have the

<    1   2   3   >