Github user galv commented on a diff in the pull request:
https://github.com/apache/spark/pull/20761#discussion_r189426163
--- Diff: docs/running-on-yarn.md ---
@@ -121,6 +121,28 @@ To use a custom metrics.properties for the application
master and executors, upd
Use lower-case suffixes, e.g. <code>k</code>, <code>m</code>,
<code>g</code>, <code>t</code>, and <code>p</code>, for kibi-, mebi-, gibi-,
tebi-, and pebibytes, respectively.
</td>
</tr>
+<tr>
+ <td><code>spark.yarn.am.resource.<resource-type></code></td>
+ <td><code>(none)</code></td>
+ <td>
+ Amount of resource to use for the YARN Application Master in client
mode.
+ In cluster mode, use
<code>spark.yarn.driver.resource.<resource-type></code> instead
--- End diff --
It would be great to show an example of what the syntax looks like. I
believe the Nvidia GPU resource type would be requested like
"spark.yarn.am.resource.yarn.io/gpu", if I remember correctly.
Also, can you add something like "This feature can be used only with Yarn
3.0+", and that some resource types require higher versions? (iirc, only Yarn
3.1 knows how to isolate GPU's with cgroups, but I'm not sure whether this
prevents you from requesting GPUs at all in Yarn 3.0).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
