Github user mridulm commented on a diff in the pull request:
https://github.com/apache/spark/pull/1218#discussion_r15827428
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1531,18 +1532,6 @@ object SparkContext extends Logging {
throw new SparkException("YARN mode not available ?", e)
}
}
- val backend = try {
- val clazz =
-
Class.forName("org.apache.spark.scheduler.cluster.YarnClusterSchedulerBackend")
- val cons = clazz.getConstructor(classOf[TaskSchedulerImpl],
classOf[SparkContext])
- cons.newInstance(scheduler,
sc).asInstanceOf[CoarseGrainedSchedulerBackend]
- } catch {
- case e: Exception => {
- throw new SparkException("YARN mode not available ?", e)
- }
- }
- scheduler.initialize(backend)
- scheduler
--- End diff --
The reflection code exists since the class can't be loaded in all env .
Tom, has this changed recently?
On 05-Aug-2014 10:27 pm, "Marcelo Vanzin" <notificati...@github.com> wrote:
> In core/src/main/scala/org/apache/spark/SparkContext.scala:
>
> > @@ -1531,18 +1532,6 @@ object SparkContext extends Logging {
> > throw new SparkException("YARN mode not available ?", e)
> > }
> > }
> > - val backend = try {
> > - val clazz =
> > -
Class.forName("org.apache.spark.scheduler.cluster.YarnClusterSchedulerBackend")
> > - val cons = clazz.getConstructor(classOf[TaskSchedulerImpl],
classOf[SparkContext])
> > - cons.newInstance(scheduler,
sc).asInstanceOf[CoarseGrainedSchedulerBackend]
> > - } catch {
> > - case e: Exception => {
> > - throw new SparkException("YARN mode not available ?", e)
> > - }
> > - }
> > - scheduler.initialize(backend)
> > - scheduler
>
> Maybe I'm missing something, but this is just removing a bunch of
> reflection code and replacing it with a single line later on (in
> YarnClusterScheduler):
>
> initialize(new YarnClusterSchedulerBackend(this, sc))
>
> This looks much, much easier to read and cleaner to me, but if you guys
> somehow feel so strongly about it, I can revert the change.
>
> â
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/1218/files#r15825749>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
