Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/3765#discussion_r23256149
--- Diff:
yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ---
@@ -43,18 +43,18 @@ object AllocationType extends Enumeration {
val HOST, RACK, ANY = Value
}
-// TODO:
-// Too many params.
-// Needs to be mt-safe
-// Need to refactor this to make it 'cleaner' ... right now, all
computation is reactive - should
-// make it more proactive and decoupled.
-
-// Note that right now, we assume all node asks as uniform in terms of
capabilities and priority
-// Refer to
http://developer.yahoo.com/blogs/hadoop/posts/2011/03/mapreduce-nextgen-scheduler/
for
-// more info on how we are requesting for containers.
-
/**
- * Acquires resources for executors from a ResourceManager and launches
executors in new containers.
+ * YarnAllocator is charged with requesting containers from the YARN
ResourceManager and deciding
+ * what to do with containers when YARN fulfills these requests.
+ *
+ * This class makes use of YARN's AMRMClient APIs. We interact with the
AMRMClient in three ways:
+ * * Making our resource needs known, which updates local bookkeeping
about containers requested.
+ * * Calling "allocate", which syncs our local container requests with the
RM, and returns any
--- End diff --
this is also a regular heartbeat
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
