Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r222392897
--- Diff: docs/running-on-yarn.md ---
@@ -121,6 +121,43 @@ To use a custom metrics.properties for the application
master and executors, upd
Use lower-case suffixes, e.g. <code>k</code>, <code>m</code>,
<code>g</code>, <code>t</code>, and <code>p</code>, for kibi-, mebi-, gibi-,
tebi-, and pebibytes, respectively.
</td>
</tr>
+<tr>
+ <td><code>spark.yarn.am.resource.{resource-type}</code></td>
+ <td><code>(none)</code></td>
+ <td>
+ Amount of resource to use for the YARN Application Master in client
mode.
+ In cluster mode, use
<code>spark.yarn.driver.resource.<resource-type></code> instead.
+ Please note that this feature can be used only with YARN 3.0+
+ For reference, see YARN Resource Model documentation:
https://hadoop.apache.org/docs/r3.0.1/hadoop-yarn/hadoop-yarn-site/ResourceModel.html
+ <p/>
+ Example:
+ To request GPU resources from YARN, use:
<code>spark.yarn.am.resource.yarn.io/gpu</code>
+ </td>
+</tr>
+<tr>
+ <td><code>spark.yarn.driver.resource.{resource-type}</code></td>
+ <td><code>(none)</code></td>
+ <td>
+ Amount of resource to use for the YARN Application Master in cluster
mode.
+ Please note that this feature can be used only with YARN 3.0+
+ For reference, see YARN Resource Model documentation:
https://hadoop.apache.org/docs/r3.0.1/hadoop-yarn/hadoop-yarn-site/ResourceModel.html
+ <p/>
+ Example:
+ To request GPU resources from YARN, use:
<code>spark.yarn.driver.resource.yarn.io/gpu</code>
+ </td>
+</tr>
+<tr>
+ <td><code>spark.yarn.executor.resource.{resource-type}</code></td>
--- End diff --
this definitely makes sense for yarn side of things, do you know what
mesos/k8s support for this? There was a lot of work with gpu supporting going
on. We had configs specific to yarn in the past and over time things move to
be more generic so if there are things we know may apply to other resource
manager perhaps we should look at generic configs
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
