[jira] [Commented] (YARN-3856) YARN shoud allocate container that is closest to the data
[ https://issues.apache.org/jira/browse/YARN-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14607945#comment-14607945 ] jaehoon ko commented on YARN-3856: -- Hi, Ravi. Thanks for your comment. I wasn't aware of YARN-18 and related discussion at the time of submission. As far as I understand, it is about adding another layer of locality - node group - between host and rack for clustesr deployed on VMs or containers. It tries to make arbitrary network topology pluggable for such needs. My patch doesn't consider an arbitrary network topology. However, according to the last comment of YARN-18, YARN community thinks that a hierarchical network is enough. In that case, I think my patch provides just-enough functionality; choosing the closest host based on network hierarchy. > YARN shoud allocate container that is closest to the data > - > > Key: YARN-3856 > URL: https://issues.apache.org/jira/browse/YARN-3856 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.7.0 > Environment: Hadoop cluster with multi-level network hierarchy >Reporter: jaehoon ko > Attachments: YARN-3856.001.patch, YARN-3856.002.patch > > > Currently, given a Container request for a host, ResourceManager allocates a > Container with following priorities (RMContainerAllocator.java): > - Requested host > - a host in the same rack as the requested host > - any host > This can lead to a sub-optimal allocation if Hadoop cluster is deployed on > multi-level networked hosts (which is typical). For example, let's suppose a > network architecture with one core switches, two aggregate switches, four ToR > switches, and 8 hosts. Each switch has two downlinks. Rack IDs of hosts are > as follows: > h1, h2: /c/a1/t1 > h3, h4: /c/a1/t2 > h5, h6: /c/a2/t3 > h7, h8: /c/a2/t4 > To allocate a container for data in h1, Hadoop first tries h1 itself, then > h2, then any of h3 ~ h8. Clearly, h3 or h4 are better than h5~h8 in terms of > network distance and bandwidth. However, current implementation choose one > from h3~h8 with equal probabilities. > This limitation is more obvious when considering hadoop clusters deployed on > VM or containers. In this case, only the VMs or containers running in the > same physical host are considered rack local, and actual rack-local hosts are > chosen with same probabilities as far hosts. > The root cause of this limitation is that RMContainerAllocator.java performs > exact matching on rack id to find a rack local host. Alternatively, we can > perform longest-prefix matching to find a closest host. Using the same > network architecture as above, with longest-prefix matching, hosts are > selected with the following priorities: > h1 > h2 > h3 or h4 > h5 or h6 or h7 or h8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3856) YARN shoud allocate container that is closest to the data
[ https://issues.apache.org/jira/browse/YARN-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606400#comment-14606400 ] Ravi Prakash commented on YARN-3856: Hi Jaehoon! Thanks for your contribution and work. Were you aware of YARN-18 ? Although that effort has stagnated, I wonder if it can be updated to support both use-cases. > YARN shoud allocate container that is closest to the data > - > > Key: YARN-3856 > URL: https://issues.apache.org/jira/browse/YARN-3856 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.7.0 > Environment: Hadoop cluster with multi-level network hierarchy >Reporter: jaehoon ko > Attachments: YARN-3856.001.patch, YARN-3856.002.patch > > > Currently, given a Container request for a host, ResourceManager allocates a > Container with following priorities (RMContainerAllocator.java): > - Requested host > - a host in the same rack as the requested host > - any host > This can lead to a sub-optimal allocation if Hadoop cluster is deployed on > multi-level networked hosts (which is typical). For example, let's suppose a > network architecture with one core switches, two aggregate switches, four ToR > switches, and 8 hosts. Each switch has two downlinks. Rack IDs of hosts are > as follows: > h1, h2: /c/a1/t1 > h3, h4: /c/a1/t2 > h5, h6: /c/a2/t3 > h7, h8: /c/a2/t4 > To allocate a container for data in h1, Hadoop first tries h1 itself, then > h2, then any of h3 ~ h8. Clearly, h3 or h4 are better than h5~h8 in terms of > network distance and bandwidth. However, current implementation choose one > from h3~h8 with equal probabilities. > This limitation is more obvious when considering hadoop clusters deployed on > VM or containers. In this case, only the VMs or containers running in the > same physical host are considered rack local, and actual rack-local hosts are > chosen with same probabilities as far hosts. > The root cause of this limitation is that RMContainerAllocator.java performs > exact matching on rack id to find a rack local host. Alternatively, we can > perform longest-prefix matching to find a closest host. Using the same > network architecture as above, with longest-prefix matching, hosts are > selected with the following priorities: > h1 > h2 > h3 or h4 > h5 or h6 or h7 or h8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3856) YARN shoud allocate container that is closest to the data
[ https://issues.apache.org/jira/browse/YARN-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605377#comment-14605377 ] jaehoon ko commented on YARN-3856: -- Can any one help me? Hadoop QA complains that pre-patch trunk has one Findbugs warning (https://builds.apache.org/job/PreCommit-YARN-Build/8374/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html). But this is not what I modified. How can I fix this? Also, there's one new Findbugs warning, but its report does not show any warning (https://builds.apache.org/job/PreCommit-YARN-Build/8374/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html). > YARN shoud allocate container that is closest to the data > - > > Key: YARN-3856 > URL: https://issues.apache.org/jira/browse/YARN-3856 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.7.0 > Environment: Hadoop cluster with multi-level network hierarchy >Reporter: jaehoon ko > Attachments: YARN-3856.001.patch, YARN-3856.002.patch > > > Currently, given a Container request for a host, ResourceManager allocates a > Container with following priorities (RMContainerAllocator.java): > - Requested host > - a host in the same rack as the requested host > - any host > This can lead to a sub-optimal allocation if Hadoop cluster is deployed on > multi-level networked hosts (which is typical). For example, let's suppose a > network architecture with one core switches, two aggregate switches, four ToR > switches, and 8 hosts. Each switch has two downlinks. Rack IDs of hosts are > as follows: > h1, h2: /c/a1/t1 > h3, h4: /c/a1/t2 > h5, h6: /c/a2/t3 > h7, h8: /c/a2/t4 > To allocate a container for data in h1, Hadoop first tries h1 itself, then > h2, then any of h3 ~ h8. Clearly, h3 or h4 are better than h5~h8 in terms of > network distance and bandwidth. However, current implementation choose one > from h3~h8 with equal probabilities. > This limitation is more obvious when considering hadoop clusters deployed on > VM or containers. In this case, only the VMs or containers running in the > same physical host are considered rack local, and actual rack-local hosts are > chosen with same probabilities as far hosts. > The root cause of this limitation is that RMContainerAllocator.java performs > exact matching on rack id to find a rack local host. Alternatively, we can > perform longest-prefix matching to find a closest host. Using the same > network architecture as above, with longest-prefix matching, hosts are > selected with the following priorities: > h1 > h2 > h3 or h4 > h5 or h6 or h7 or h8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3856) YARN shoud allocate container that is closest to the data
[
https://issues.apache.org/jira/browse/YARN-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605339#comment-14605339
]
Hadoop QA commented on YARN-3856:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 55s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 35s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 38s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 20s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 2m 42s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 9m 0s | Tests passed in
hadoop-mapreduce-client-app. |
| {color:green}+1{color} | yarn tests | 1m 57s | Tests passed in
hadoop-yarn-common. |
| | | 51m 44s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-common |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12742477/YARN-3856.002.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / a95d39f |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/8374/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/8374/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8374/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8374/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8374/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8374/console |
This message was automatically generated.
> YARN shoud allocate container that is closest to the data
> -
>
> Key: YARN-3856
> URL: https://issues.apache.org/jira/browse/YARN-3856
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.7.0
> Environment: Hadoop cluster with multi-level network hierarchy
>Reporter: jaehoon ko
> Attachments: YARN-3856.001.patch, YARN-3856.002.patch
>
>
> Currently, given a Container request for a host, ResourceManager allocates a
> Container with following priorities (RMContainerAllocator.java):
> - Requested host
> - a host in the same rack as the requested host
> - any host
> This can lead to a sub-optimal allocation if Hadoop cluster is deployed on
> multi-level networked hosts (which is typical). For example, let's suppose a
> network architecture with one core switches, two aggregate switches, four ToR
> switches, and 8 hosts. Each switch has two downlinks. Rack IDs of hosts are
> as follows:
> h1, h2: /c/a1/t1
> h3, h4: /c/a1/t2
> h5, h6: /c/a2/t3
> h7, h8: /c/a2/t4
> To allocate a container for data in h1, Hadoop first tries h1 itself, then
> h2, then any of h3 ~ h8. Clearly, h3 or h4 are better than h5~h8 in terms of
> network distance and bandwidth. However, current implementation choose one
> from h3~h8 with equal probabilities.
> This limitation is more obvious when considering hadoop clusters deployed on
> VM or containers. In this case, only the VMs or containers running in the
> same physical host are considered rack local, and actual rack-local hosts are
> chosen with same probabilities as far hosts.
> The root cause of this limitation is that RMContainerAllocator.java performs
> exact matching on rack id to find a rack local host. Alternatively, we can
> perform longest-prefix matching to find a closest host. Using the same
> network architecture as ab
[jira] [Commented] (YARN-3856) YARN shoud allocate container that is closest to the data
[
https://issues.apache.org/jira/browse/YARN-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602550#comment-14602550
]
Hadoop QA commented on YARN-3856:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 54s | Pre-patch trunk has 1 extant
Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 35s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 36s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 16s | The applied patch generated
19 new checkstyle issues (total was 0, now 19). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 2m 41s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | mapreduce tests | 9m 3s | Tests passed in
hadoop-mapreduce-client-app. |
| {color:green}+1{color} | yarn tests | 1m 56s | Tests passed in
hadoop-yarn-common. |
| | | 51m 33s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-common |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12742067/YARN-3856.001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 8ef07f7 |
| Pre-patch Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/8356/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/8356/artifact/patchprocess/diffcheckstylehadoop-yarn-common.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/8356/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
|
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8356/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8356/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8356/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8356/console |
This message was automatically generated.
> YARN shoud allocate container that is closest to the data
> -
>
> Key: YARN-3856
> URL: https://issues.apache.org/jira/browse/YARN-3856
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.7.0
> Environment: Hadoop cluster with multi-level network hierarchy
>Reporter: jaehoon ko
> Attachments: YARN-3856.001.patch
>
>
> Currently, given a Container request for a host, ResourceManager allocates a
> Container with following priorities (RMContainerAllocator.java):
> - Requested host
> - a host in the same rack as the requested host
> - any host
> This can lead to a sub-optimal allocation if Hadoop cluster is deployed on
> multi-level networked hosts (which is typical). For example, let's suppose a
> network architecture with one core switches, two aggregate switches, four ToR
> switches, and 8 hosts. Each switch has two downlinks. Rack IDs of hosts are
> as follows:
> h1, h2: /c/a1/t1
> h3, h4: /c/a1/t2
> h5, h6: /c/a2/t3
> h7, h8: /c/a2/t4
> To allocate a container for data in h1, Hadoop first tries h1 itself, then
> h2, then any of h3 ~ h8. Clearly, h3 or h4 are better than h5~h8 in terms of
> network distance and bandwidth. However, current implementation choose one
> from h3~h8 with equal probabilities.
> This limitation is more obvious when considering hadoop clusters deployed on
> VM or containers. In this case, only the VMs or containers running in the
> same physical host are considered rack local, and actual rack-local hosts are
> chosen with same probabilities as far hosts.
> The root cause of this limitation is that RMContainerAllocator.java performs
> exact matching on rack id
