tgravescs commented on a change in pull request #25047:
[WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone
URL: https://github.com/apache/spark/pull/25047#discussion_r308369123
##########
File path: docs/spark-standalone.md
##########
@@ -243,6 +243,33 @@ SPARK_MASTER_OPTS supports the following system
properties:
receives no heartbeats.
</td>
</tr>
+<tr>
+ <td><code>spark.worker.resource.{resourceName}.amount</code></td>
+ <td>(none)</td>
+ <td>
+ Amount of a particular resource to use on the worker.
+ </td>
+</tr>
+<tr>
+ <td><code>spark.worker.resource.{resourceName}.discoveryScript</code></td>
+ <td>(none)</td>
+ <td>
+ Path to resource discovery script, which is used to find a particular
resource while worker starting up.
+ And the output of the script should be formatted like <code>{"name":
"gpu", "addresses": ["0","1"]}</code>.
+ </td>
+</tr>
+<tr>
+ <td><code>spark.worker.resourcesFile</code></td>
+ <td>(none)</td>
+ <td>
+ Path to resources file which is used to find various resources while
worker starting up.
+ The content of resources file should be formatted like
<code>[[{"id":{"componentName":
+ "spark.worker","resourceName":"gpu"},"addresses":["0","1","2"]}]]</code>.
Review comment:
Actually thinking about this more, I guess if our comment references
ResourceAllocation they won't be able to find it in the scala docs which
wouldn't be very convenient. And I guess since the end user would be using it
as an api for the resourcesFile we should make it public then. I would put an
@evolving tag on it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
