Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2373#discussion_r17509009
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ---
@@ -121,7 +124,7 @@ private[yarn] abstract class YarnAllocator(
logDebug("Empty allocation request ...")
}
- val allocateResponse = allocateContainers(missing)
+ val allocateResponse = allocateContainers(missing, executorsPending)
--- End diff --
I think it's safe to expose the variable as long as it does what its name
suggests. We actually already do this elsewhere in the same class for similar
variables (e.g. `getNumExecutorRunning` or `getNumExecutorFailed`). In our
case, we can't make just those changes because we increment
`numPendingAllocate` before we `allocateContainers`, but we can easily work
around that as you mentioned. These allocation requests aren't technically
pending until we've submitted them, so I think it makes sense to increment
`numPendingAllocate` afterwards.
Anyway, just a minor suggestion since this is an internal API. It's fine to
leave it as is if you prefer.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
