wbo4958 commented on code in PR #43494:
URL: https://github.com/apache/spark/pull/43494#discussion_r1390665985
##########
core/src/main/scala/org/apache/spark/resource/ResourceUtils.scala:
##########
@@ -170,16 +170,16 @@ private[spark] object ResourceUtils extends Logging {
// integer amount and the number of slots per address. For instance, if the
amount is 0.5,
// the we get (1, 2) back out. This indicates that for each 1 address, it
has 2 slots per
// address, which allows you to put 2 tasks on that address. Note if amount
is greater
- // than 1, then the number of slots per address has to be 1. This would
indicate that a
+ // than 1, then the number of parts per address has to be 1. This would
indicate that a
// would have multiple addresses assigned per task. This can be used for
calculating
// the number of tasks per executor -> (executorAmount * numParts) /
(integer amount).
// Returns tuple of (integer amount, numParts)
def calculateAmountAndPartsForFraction(doubleAmount: Double): (Int, Int) = {
- val parts = if (doubleAmount <= 0.5) {
+ val parts = if (doubleAmount <= 1.0) {
Review Comment:
I added the check in the ResourceProfile which requires the task.amount to
be in (0, 0.5] or a whole number, while in the TaskResourceProfile which
requires to be (0, 1.0] or a whole number
##########
core/src/main/scala/org/apache/spark/resource/TaskResourceRequest.scala:
##########
@@ -37,8 +37,8 @@ import org.apache.spark.annotation.{Evolving, Since}
class TaskResourceRequest(val resourceName: String, val amount: Double)
extends Serializable {
- assert(amount <= 0.5 || amount % 1 == 0,
- s"The resource amount ${amount} must be either <= 0.5, or a whole number.")
+ assert(amount <= 1.0 || amount % 1 == 0,
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
