[jira] [Updated] (YARN-1643) Make ContainersMonitor can support change monitoring size of an allocated container in NM side
[ https://issues.apache.org/jira/browse/YARN-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-1643: - Fix Version/s: (was: 3.0.0) 2.8.0 > Make ContainersMonitor can support change monitoring size of an allocated > container in NM side > -- > > Key: YARN-1643 > URL: https://issues.apache.org/jira/browse/YARN-1643 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Wangda Tan >Assignee: MENG DING > Fix For: 2.8.0 > > Attachments: YARN-1643-YARN-1197.4.patch, > YARN-1643-YARN-1197.5.patch, YARN-1643-YARN-1197.6.patch, > YARN-1643-YARN-1197.7.patch, YARN-1643.1.patch, YARN-1643.2.patch, > YARN-1643.3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-1643) Make ContainersMonitor can support change monitoring size of an allocated container in NM side
[ https://issues.apache.org/jira/browse/YARN-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-1643: - Fix Version/s: (was: YARN-1197) 3.0.0 > Make ContainersMonitor can support change monitoring size of an allocated > container in NM side > -- > > Key: YARN-1643 > URL: https://issues.apache.org/jira/browse/YARN-1643 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Reporter: Wangda Tan >Assignee: MENG DING > Fix For: 3.0.0 > > Attachments: YARN-1643-YARN-1197.4.patch, > YARN-1643-YARN-1197.5.patch, YARN-1643-YARN-1197.6.patch, > YARN-1643-YARN-1197.7.patch, YARN-1643.1.patch, YARN-1643.2.patch, > YARN-1643.3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-1643) Make ContainersMonitor can support change monitoring size of an allocated container in NM side
[
https://issues.apache.org/jira/browse/YARN-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
MENG DING updated YARN-1643:
Attachment: YARN-1643-YARN-1197.7.patch
Attaching latest patch which addressed the following:
* Make {{trackingContainers}} a ConcurrentHashMap, and remove
{{containersToBeRemoved}} and {{containersToBeAdded}} and corresponding logic.
Containers are directly added to/removed from/updated in {{trackingContainers}}
when corresponding events are received.
* Synchronize getters and setters in {{ProcessTreeInfo}} with regard to the
vmemLimit/pmemLimit/cpuVcores fields.
* Previous patch didn't handle container metrics update for container resize.
Add it and extract container metrics logic into a common function.
* Add relevant test cases
Make ContainersMonitor can support change monitoring size of an allocated
container in NM side
--
Key: YARN-1643
URL: https://issues.apache.org/jira/browse/YARN-1643
Project: Hadoop YARN
Issue Type: Sub-task
Components: nodemanager
Reporter: Wangda Tan
Assignee: MENG DING
Attachments: YARN-1643-YARN-1197.4.patch,
YARN-1643-YARN-1197.5.patch, YARN-1643-YARN-1197.6.patch,
YARN-1643-YARN-1197.7.patch, YARN-1643.1.patch, YARN-1643.2.patch,
YARN-1643.3.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-1643) Make ContainersMonitor can support change monitoring size of an allocated container in NM side
[ https://issues.apache.org/jira/browse/YARN-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] MENG DING updated YARN-1643: Attachment: YARN-1643-YARN-1197.6.patch Update the patch to fix build issue as YARN-1197 branch has been rebased. Make ContainersMonitor can support change monitoring size of an allocated container in NM side -- Key: YARN-1643 URL: https://issues.apache.org/jira/browse/YARN-1643 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Wangda Tan Assignee: MENG DING Attachments: YARN-1643-YARN-1197.4.patch, YARN-1643-YARN-1197.5.patch, YARN-1643-YARN-1197.6.patch, YARN-1643.1.patch, YARN-1643.2.patch, YARN-1643.3.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-1643) Make ContainersMonitor can support change monitoring size of an allocated container in NM side
[ https://issues.apache.org/jira/browse/YARN-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] MENG DING updated YARN-1643: Attachment: YARN-1643-YARN-1197.4.patch We have decided to update resource synchronously in YARN-3867 for now. First upload a patch (YARN-1643.4.patch) that does container monitor and resource enforcement asynchronously for reference, in case we need to support asynchronous resource update in the future. Make ContainersMonitor can support change monitoring size of an allocated container in NM side -- Key: YARN-1643 URL: https://issues.apache.org/jira/browse/YARN-1643 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Wangda Tan Assignee: MENG DING Attachments: YARN-1643-YARN-1197.4.patch, YARN-1643.1.patch, YARN-1643.2.patch, YARN-1643.3.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-1643) Make ContainersMonitor can support change monitoring size of an allocated container in NM side
[ https://issues.apache.org/jira/browse/YARN-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] MENG DING updated YARN-1643: Attachment: (was: YARN-3867-YARN-1197.5.patch) Make ContainersMonitor can support change monitoring size of an allocated container in NM side -- Key: YARN-1643 URL: https://issues.apache.org/jira/browse/YARN-1643 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Wangda Tan Assignee: MENG DING Attachments: YARN-1643-YARN-1197.4.patch, YARN-1643-YARN-1197.5.patch, YARN-1643.1.patch, YARN-1643.2.patch, YARN-1643.3.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-1643) Make ContainersMonitor can support change monitoring size of an allocated container in NM side
[ https://issues.apache.org/jira/browse/YARN-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] MENG DING updated YARN-1643: Attachment: YARN-1643-YARN-1197.5.patch Make ContainersMonitor can support change monitoring size of an allocated container in NM side -- Key: YARN-1643 URL: https://issues.apache.org/jira/browse/YARN-1643 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Wangda Tan Assignee: MENG DING Attachments: YARN-1643-YARN-1197.4.patch, YARN-1643-YARN-1197.5.patch, YARN-1643.1.patch, YARN-1643.2.patch, YARN-1643.3.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-1643) Make ContainersMonitor can support change monitoring size of an allocated container in NM side
[
https://issues.apache.org/jira/browse/YARN-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
MENG DING updated YARN-1643:
Attachment: YARN-3867-YARN-1197.5.patch
Upload the patch that supports *synchronous* resource update in
{{ContainersMonitorImpl}}.
There are a number of changes:
* Extract out the code from the {{MonitoringThread.run}} and put it into a
common function {{enforceResourceLimits}}. This function will be called by both
ContainersMonitorImpl's monitoring thread, and the ContainerManagerImpl thread,
so it needs to be synchronized (in order to guard access to the
{{trackingContainers}} object).
* Call container.setResource directly to update container resource.
* Add {{TestContainerManager#testChangeContainerResource}} and
{{TestContainersMonitorResourceChange}} test cases.
Make ContainersMonitor can support change monitoring size of an allocated
container in NM side
--
Key: YARN-1643
URL: https://issues.apache.org/jira/browse/YARN-1643
Project: Hadoop YARN
Issue Type: Sub-task
Components: nodemanager
Reporter: Wangda Tan
Assignee: MENG DING
Attachments: YARN-1643-YARN-1197.4.patch,
YARN-1643-YARN-1197.5.patch, YARN-1643.1.patch, YARN-1643.2.patch,
YARN-1643.3.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-1643) Make ContainersMonitor can support change monitoring size of an allocated container in NM side
[ https://issues.apache.org/jira/browse/YARN-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] MENG DING updated YARN-1643: Attachment: YARN-1643.3.patch Updated the patch to resolve the conflicts with YARN-1012 based on the latest rebase. Make ContainersMonitor can support change monitoring size of an allocated container in NM side -- Key: YARN-1643 URL: https://issues.apache.org/jira/browse/YARN-1643 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Wangda Tan Assignee: MENG DING Attachments: YARN-1643.1.patch, YARN-1643.2.patch, YARN-1643.3.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-1643) Make ContainersMonitor can support change monitoring size of an allocated container in NM side
[
https://issues.apache.org/jira/browse/YARN-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
MENG DING updated YARN-1643:
Attachment: YARN-1643.2.patch
Attach patch for {{ContainersMonitorImpl}} for review.
Currently the {{ContainersMonitorImpl}} starts up one monitoring thread, and by
default checks started and stopped containers every 3 seconds.
For resized containers, we cannot use the same logic as the delay is
unacceptable.
A wait/notify mechanism is implemented such that as soon as a
{{CHANGE_MONITORING_CONTAINER_RESOURCE}} event is received, the monitor cycle
is immediately triggered.
Make ContainersMonitor can support change monitoring size of an allocated
container in NM side
--
Key: YARN-1643
URL: https://issues.apache.org/jira/browse/YARN-1643
Project: Hadoop YARN
Issue Type: Sub-task
Components: nodemanager
Reporter: Wangda Tan
Assignee: MENG DING
Attachments: YARN-1643.1.patch, YARN-1643.2.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-1643) Make ContainersMonitor can support change monitoring size of an allocated container in NM side
[ https://issues.apache.org/jira/browse/YARN-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-1643: - Summary: Make ContainersMonitor can support change monitoring size of an allocated container in NM side (was: [YARN-1197] Make ContainersMonitor can support change monitoring size of an allocated container in NM side) Make ContainersMonitor can support change monitoring size of an allocated container in NM side -- Key: YARN-1643 URL: https://issues.apache.org/jira/browse/YARN-1643 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Wangda Tan Assignee: Wangda Tan (No longer used) Attachments: YARN-1643.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-1643) Make ContainersMonitor can support change monitoring size of an allocated container in NM side
[ https://issues.apache.org/jira/browse/YARN-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-1643: - Assignee: Wangda Tan (was: Wangda Tan (No longer used)) Make ContainersMonitor can support change monitoring size of an allocated container in NM side -- Key: YARN-1643 URL: https://issues.apache.org/jira/browse/YARN-1643 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Wangda Tan Assignee: Wangda Tan Attachments: YARN-1643.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-1643) Make ContainersMonitor can support change monitoring size of an allocated container in NM side
[ https://issues.apache.org/jira/browse/YARN-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-1643: - Assignee: (was: Wangda Tan) Make ContainersMonitor can support change monitoring size of an allocated container in NM side -- Key: YARN-1643 URL: https://issues.apache.org/jira/browse/YARN-1643 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Wangda Tan Attachments: YARN-1643.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
