[
https://issues.apache.org/jira/browse/YARN-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15983668#comment-15983668
]
Wangda Tan commented on YARN-5983:
----------------------------------
Thanks [~tangzhankun], several questions / comments:
Regarding to exclusive/non-exclusive resource, I think the best way to solve
the problem is use node attribute (YARN-3409). And instead of naming it
"exclusive resource", I would prefer to call it scheduling perference. To me
all resource are exclusive because nobody can use it if it is using by anyone
else.
I'm not quite sure about this:
bq. No NM side resource management of FPGA resource. For instance, dynamically
resource discovery, monitoring and preparation before container launch.
Could you add more details?
And also:
bq. 4. AM set the IP UUID/name in container environment and sends requests to
NM to launch the allocated containers.
Should AM agnostic to IP UUID/name for allocated container? Ideally AM should
only care about amount of each resource type (like FPGA), and NM will decide
how to proceed. It will be helpful if you can share more details about what is
required to run a FPGA application.
LocalResourceScheduler in the doc will be placed inside implementation of
ResourceHandler, correct? IIUC, it is specific to FPGA, let me know if I
understand wrong.
> [Umbrella] Support for FPGA as a Resource in YARN
> -------------------------------------------------
>
> Key: YARN-5983
> URL: https://issues.apache.org/jira/browse/YARN-5983
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: yarn
> Reporter: Zhankun Tang
> Assignee: Zhankun Tang
> Attachments: YARN-5983-Support-FPGA-resource-on-NM-side_v1.pdf
>
>
> As various big data workload running on YARN, CPU will no longer scale
> eventually and heterogeneous systems will become more important. ML/DL is a
> rising star in recent years, applications focused on these areas have to
> utilize GPU or FPGA to boost performance. Also, hardware vendors such as
> Intel also invest in such hardware. It is most likely that FPGA will become
> popular in data centers like CPU in the near future.
> So YARN as a resource managing and scheduling system, would be great to
> evolve to support this. This JIRA proposes FPGA to be a first-class citizen.
> The changes roughly includes:
> 1. FPGA resource detection and heartbeat
> 2. Scheduler changes
> 3. FPGA related preparation and isolation before launch container
> We know that YARN-3926 is trying to extend current resource model. But still
> we can leave some FPGA related discussion here
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
