tgravescs commented on a change in pull request #31496:
URL: https://github.com/apache/spark/pull/31496#discussion_r578494381
##########
File path:
core/src/main/scala/org/apache/spark/resource/ResourceProfileBuilder.scala
##########
@@ -26,17 +26,19 @@ import org.apache.spark.annotation.{Evolving, Since}
/**
- * Resource profile builder to build a Resource profile to associate with an
RDD.
- * A ResourceProfile allows the user to specify executor and task requirements
for an RDD
- * that will get applied during a stage. This allows the user to change the
resource
+ * Resource profile builder to build a [[ResourceProfile]] to associate with
an RDD.
+ * A [[ResourceProfile]] allows the user to specify executor and task resource
requirements
+ * for an RDD that will get applied during a stage. This allows the user to
change the resource
* requirements between stages.
*
*/
@Evolving
@Since("3.1.0")
class ResourceProfileBuilder() {
+ // Task resource requests that specified by users, mapped from resource name
to the request.
private val _taskResources = new ConcurrentHashMap[String,
TaskResourceRequest]()
+ // Executor resource requests that specified by users, mapped from resource
name to the request.
Review comment:
nit remove "that"
##########
File path:
core/src/main/scala/org/apache/spark/resource/ResourceProfileBuilder.scala
##########
@@ -26,17 +26,19 @@ import org.apache.spark.annotation.{Evolving, Since}
/**
- * Resource profile builder to build a Resource profile to associate with an
RDD.
- * A ResourceProfile allows the user to specify executor and task requirements
for an RDD
- * that will get applied during a stage. This allows the user to change the
resource
+ * Resource profile builder to build a [[ResourceProfile]] to associate with
an RDD.
+ * A [[ResourceProfile]] allows the user to specify executor and task resource
requirements
+ * for an RDD that will get applied during a stage. This allows the user to
change the resource
* requirements between stages.
*
*/
@Evolving
@Since("3.1.0")
class ResourceProfileBuilder() {
+ // Task resource requests that specified by users, mapped from resource name
to the request.
Review comment:
remove "that"
##########
File path:
core/src/main/scala/org/apache/spark/resource/TaskResourceRequest.scala
##########
@@ -20,11 +20,15 @@ package org.apache.spark.resource
import org.apache.spark.annotation.{Evolving, Since}
/**
- * A task resource request. This is used in conjunction with the
ResourceProfile to
+ * A task resource request. This is used in conjunction with the
[[ResourceProfile]] to
* programmatically specify the resources needed for an RDD that will be
applied at the
* stage level.
*
- * Use TaskResourceRequests class as a convenience API.
+ * Use [[TaskResourceRequests]] class as a convenience API.
+ *
+ * @param resourceName Resource name
+ * @param amount Expected amount of the resource, must be either less than or
equal to 0.5
Review comment:
perhaps should clarify to be similar to text in TaskResourceRequests:
@param amount Amount requesting as a Double to support fractional resource
requests.
Valid values are less than or equal to 0.5 or whole
numbers. This essentially
lets you configure X number of tasks to run on a single
resource,
ie amount equals 0.5 translates into 2 tasks per resource
address.
##########
File path:
core/src/main/scala/org/apache/spark/resource/ResourceProfileBuilder.scala
##########
@@ -54,11 +56,21 @@ class ResourceProfileBuilder() {
_executorResources.asScala.asJava
}
+ /**
+ * Add executor resource requests
+ * @param requests The detailed executor resource requests, see
[[ExecutorResourceRequests]]
+ * @return this.type
Review comment:
perhaps better to say return "This ResourceProfileBuilder"
##########
File path:
core/src/main/scala/org/apache/spark/resource/ResourceProfileBuilder.scala
##########
@@ -54,11 +56,21 @@ class ResourceProfileBuilder() {
_executorResources.asScala.asJava
}
+ /**
+ * Add executor resource requests
+ * @param requests The detailed executor resource requests, see
[[ExecutorResourceRequests]]
+ * @return this.type
+ */
def require(requests: ExecutorResourceRequests): this.type = {
_executorResources.putAll(requests.requests.asJava)
this
}
+ /**
+ * Add task resource requests
+ * @param requests The detailed task resource requests, see
[[TaskResourceRequest]]
+ * @return this.type
Review comment:
perhaps better to say return "This ResourceProfileBuilder"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
