Hi Jo, Thanks for the links, I would expected the properties to be in scheduler properties but I need to double check.
I'll be looking into these problems this week. Tim On Tue, Nov 17, 2015 at 10:28 AM, Jo Voordeckers <jo.voordeck...@gmail.com> wrote: > On Tue, Nov 17, 2015 at 5:16 AM, Iulian Dragoș <iulian.dra...@typesafe.com> > wrote: >> >> I think it actually tries to send all properties as part of >> `SPARK_EXECUTOR_OPTS`, which may not be everything that's needed: >> >> >> https://github.com/jayv/spark/blob/mesos_cluster_params/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala#L375-L377 >> > > Aha that's interesting, I overlooked that line, I'll debug some more today > because I know for sure that those options don't make it onto the > commandline when I was running it in my debugger. > >> >> Can you please open a Jira ticket and describe also the symptoms? This >> might be related, or the same issue: SPARK-11280 and also SPARK-11327 > > > SPARK-11327 is exactly my problem, but I don't run docker. > > - Jo > >> On Tue, Nov 17, 2015 at 2:46 AM, Jo Voordeckers <jo.voordeck...@gmail.com> >> wrote: >>> >>> >>> Hi all, >>> >>> I'm running the mesos cluster dispatcher, however when I submit jobs with >>> things like jvm args, classpath order and UI port aren't added to the >>> commandline executed by the mesos scheduler. In fact it only cares about the >>> class, jar and num cores/mem. >>> >>> >>> https://github.com/jayv/spark/blob/mesos_cluster_params/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala#L412-L424 >>> >>> I've made an attempt at adding a few of the args that I believe are >>> useful to the MesosClusterScheduler class, which seems to solve my problem. >>> >>> Please have a look: >>> >>> https://github.com/apache/spark/pull/9752 >>> >>> Thanks >>> >>> - Jo Voordeckers >>> >> >> >> >> -- >> >> -- >> Iulian Dragos >> >> ------ >> Reactive Apps on the JVM >> www.typesafe.com >> > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
