[
https://issues.apache.org/jira/browse/YARN-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449284#comment-16449284
]
Sunil G commented on YARN-8200:
-------------------------------
Hi [~jhung]
YARN-3926 has two parts of the work. Supporting multiple resource types and
resource profiles. Due to the complex nature of resource profiles, Hadoop-3.0
still contains only resource types which is a part of YARN-3926. Moreover there
is a chance that CS performance could be impacted. For 3.0 and 3.1, many core
scheduler part changes were there compared to branch-2. It was more easy as
whole resource types development happened in trunk based code hence we could
cross check the impacted areas well. In branch-2, there may be more chances of
performance impact as code difference to 3.0 and branch-2.10 are bit of huge.
> Backport resource types/GPU features to branch-2
> ------------------------------------------------
>
> Key: YARN-8200
> URL: https://issues.apache.org/jira/browse/YARN-8200
> Project: Hadoop YARN
> Issue Type: Task
> Reporter: Jonathan Hung
> Assignee: Jonathan Hung
> Priority: Major
>
> Currently we have a need for GPU scheduling on our YARN clusters to support
> deep learning workloads. However, our main production clusters are running
> older versions of branch-2 (2.7 in our case). To prevent supporting too many
> very different hadoop versions across multiple clusters, we would like to
> backport the resource types/resource profiles feature to branch-2, as well as
> the GPU specific support.
>
> We have done a trial backport of YARN-3926 and some miscellaneous patches in
> YARN-7069 based on issues we uncovered, and the backport was fairly smooth.
> We also did a trial backport of most of YARN-6223 (sans docker support).
>
> Regarding the backports, perhaps we can do the development in a feature
> branch and then merge to branch-2 when ready.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
