I quickly went through the code,
In ExecutorBackend, we build the actor system with
// Create SparkEnv using properties we fetched from the driver.
val driverConf = new SparkConf().setAll(props)
val env = SparkEnv.createExecutorEnv(
driverConf, executorId, hostname, port, cores, isLocal = false)
props is the spark properties fetched from driver,
In Driver side, we start driver actor with
val properties = new ArrayBuffer[(String, String)]
for ((key, value) <- scheduler.sc.conf.getAll) {
if (key.startsWith("spark.")) {
properties += ((key, value))
}
}
// TODO (prashant) send conf instead of properties
driverActor = actorSystem.actorOf(
Props(new DriverActor(properties)), name =
CoarseGrainedSchedulerBackend.ACTOR_NAME)
This “properties” is the stuff being sent to executor. It seems that we only
admit the properties starting with “spark.”
It seems that we cannot pass akka.* to executor?
Hi, Josh, would you mind giving some hints, as you created and closed the JIRA?
Best,
--
Nan Zhu
On Wednesday, January 14, 2015 at 6:19 PM, Nan Zhu wrote:
> Hi, Ted,
>
> Thanks
>
> I know how to set in Akka’s context, my question is just how to pass this
> aka.loglevel=DEBUG to Spark’s actor system
>
> Best,
> --
> Nan Zhu
> http://codingcat.me
>
>
> On Wednesday, January 14, 2015 at 6:09 PM, Ted Yu wrote:
>
> > I assume you have looked at:
> >
> > http://doc.akka.io/docs/akka/2.0/scala/logging.html
> > http://doc.akka.io/docs/akka/current/additional/faq.html (Debugging, last
> > question)
> >
> > Cheers
> >
> > On Wed, Jan 14, 2015 at 2:55 PM, Nan Zhu <[email protected]
> > (mailto:[email protected])> wrote:
> > > Hi, all
> > >
> > > though https://issues.apache.org/jira/browse/SPARK-609 was closed, I’m
> > > still unclear about how to enable debug level log output in Spark’s actor
> > > system
> > >
> > > Anyone can give the suggestion?
> > >
> > > BTW, I think we need to document it on somewhere, as the user who writes
> > > actor-based receiver of spark streaming, (like me), usually needs the
> > > detailed log for debugging….
> > >
> > > Best,
> > >
> > > --
> > > Nan Zhu
> > > http://codingcat.me
> > >
> > >
> >
> >
> >
>
