wbo4958 commented on code in PR #43494:
URL: https://github.com/apache/spark/pull/43494#discussion_r1375804675
##########
core/src/main/scala/org/apache/spark/deploy/master/WorkerInfo.scala:
##########
@@ -28,12 +29,24 @@ private[spark] case class WorkerResourceInfo(name: String,
addresses: Seq[String
override protected def resourceName = this.name
override protected def resourceAddresses = this.addresses
- override protected def slotsPerAddress: Int = 1
+ /**
+ * Acquire the resources.
+ * @param amount How many addresses are requesting.
+ * @return ResourceInformation
+ */
def acquire(amount: Int): ResourceInformation = {
- val allocated = availableAddrs.take(amount)
- acquire(allocated)
- new ResourceInformation(resourceName, allocated.toArray)
+
+ var count = amount
+ val allocated: mutable.HashMap[String, Double] = mutable.HashMap.empty
+ for (address <- availableAddrs if count > 0) {
+ if (addressAvailabilityMap(address) == RESOURCE_TOTAL_AMOUNT) {
+ allocated(address) = 1.0
Review Comment:
Good suggestion. I modified the code so that it no longer checks if it's a
complete GPU, as any address from the availableAddrs implies it must be a full
GPU. Besides that, I can prevent the need to grant access to the
addressAvailabilityMap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
