[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated YARN-443: -- Fix Version/s: (was: 3.0.0) allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves Fix For: 0.23.7, 2.0.4-alpha Attachments: YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-2.patch, YARN-443-branch-2.patch, YARN-443-branch-2.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-443: --- Attachment: YARN-443-branch-2.patch YARN-443-branch-0.23.patch patches for branch-2 and branch-0.23. allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves Attachments: YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-2.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-443: --- Attachment: YARN-443.patch trunk patch. Note that I have nice -n applying to the winutils.exe call to, but unfortunately haven't tested it as I don't have a windows box. Bikas if you have cycles to test I would appreciate it, if not I can remove the windows part and file a follow on jira. allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves Attachments: YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-2.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-443: --- Attachment: YARN-443.patch added a couple of unit tests. Still need to update the branch-2 and branch-0.23 patches to include the tests. allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves Attachments: YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-2.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-443: --- Attachment: YARN-443.patch All patches updated with changes from review and all include tests. allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves Attachments: YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-2.patch, YARN-443-branch-2.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-443: --- Attachment: YARN-443-branch-2.patch YARN-443-branch-0.23.patch allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves Attachments: YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-2.patch, YARN-443-branch-2.patch, YARN-443-branch-2.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-443: --- Attachment: YARN-443.patch added timeouts to tests allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves Attachments: YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-2.patch, YARN-443-branch-2.patch, YARN-443-branch-2.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-443: --- Attachment: YARN-443.patch allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves Attachments: YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-0.23.patch, YARN-443-branch-2.patch, YARN-443-branch-2.patch, YARN-443-branch-2.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch, YARN-443.patch It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-443: --- Attachment: YARN-443.patch complete patch where code is in container-executor.c. Attaching here since it was done for reference. allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves Attachments: YARN-443.patch, YARN-443.patch It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-443: --- Attachment: YARN-443.patch I decided to go with the nice method as it should apply across all the platforms and I don't see setting it to a negative value that useful. If its really needed you can run your nodemanager with root permissions. If the windows port isn't using cygwin or you think it will cause problems let me know. The config in the patch is named: container-executor.os.sched.priority.adjustment It basically does a nice -n using the config before launching the container. Attaching the branch-0.23 patch which I've tested. I need to create the trunk version with the windows stuff included. allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves Attachments: YARN-443.patch, YARN-443.patch, YARN-443.patch It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-443: --- Attachment: (was: YARN-443.patch) allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves Attachments: YARN-443-branch-0.23.patch, YARN-443.patch, YARN-443.patch It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-443: --- Attachment: YARN-443-branch-0.23.patch allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves Attachments: YARN-443-branch-0.23.patch, YARN-443.patch, YARN-443.patch It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated YARN-443: --- Attachment: YARN-443.patch allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves Attachments: YARN-443.patch It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-443: Summary: allow OS scheduling priority of NM to be different than the containers it launches (was: allow OS scheduling priority of NM to be different then the containers it launches) allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves It would be nice if we could have the nodemanager run at a different OS scheduling priority then the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-443) allow OS scheduling priority of NM to be different than the containers it launches
[ https://issues.apache.org/jira/browse/YARN-443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-443: Description: It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. was: It would be nice if we could have the nodemanager run at a different OS scheduling priority then the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. allow OS scheduling priority of NM to be different than the containers it launches -- Key: YARN-443 URL: https://issues.apache.org/jira/browse/YARN-443 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.0.3-alpha, 0.23.6 Reporter: Thomas Graves Assignee: Thomas Graves It would be nice if we could have the nodemanager run at a different OS scheduling priority than the containers so that you can still communicate with the nodemanager if the containers out of control. On linux we could launch the nodemanager at a higher priority, but then all the containers it launches would also be at that higher priority, so we need a way for the container executor to launch them at a lower priority. I'm not sure how this applies to windows if at all. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
