tgravescs commented on a change in pull request #25047:
[WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone
URL: https://github.com/apache/spark/pull/25047#discussion_r310181649
##########
File path: docs/spark-standalone.md
##########
@@ -243,6 +243,36 @@ SPARK_MASTER_OPTS supports the following system
properties:
receives no heartbeats.
</td>
</tr>
+<tr>
+ <td><code>spark.worker.resource.{resourceName}.amount</code></td>
+ <td>(none)</td>
+ <td>
+ Amount of a particular resource to use on the worker.
+ </td>
+</tr>
+<tr>
+ <td><code>spark.worker.resource.{resourceName}.discoveryScript</code></td>
+ <td>(none)</td>
+ <td>
+ Path to resource discovery script, which is used to find a particular
resource while worker starting up.
+ And the output of the script should be formatted like the
<code>ResourceInformation</code> class.
+ When <code>spark.resources.coordinate.enable</code> is off, discovery
script must assign different
+ resource addresses for workers/drivers(client only) to avoid resource
conflict.
+ </td>
+</tr>
+<tr>
+ <td><code>spark.worker.resourcesFile</code></td>
+ <td>(none)</td>
+ <td>
+ Path to resources file which is used to find various resources while
worker starting up.
+ The content of resources file should be formatted like the
<code>ResourceAllocation</code> class.
+ Unlike discovery script, resources file should always assign different
resource addresses for
+ workers/drivers(client only) whether
<code>spark.resources.coordinate.enable</code> is off or not.
+ If a particular resource required by amount config does not found in
resources file, discovery
+ script would be used to find that resource. If discovery script also does
not config for
Review comment:
If the discovery script also does not find the resources, the worker will
fail to start up.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
