I have been able to run spark submission in docker container (HOST network)
through Marathon on mesos and target to Mesos cluster (zk address) for at least
Spark 1.6, 1.5.2 over Mesos 0.26, 0.27.
I do need to define SPARK_PUBLIC_DNS and SPARK_LOCAL_IP so that the spark
driver can announce the
I finally isolated the issue to be related to the ActorSystem I reuse from
SparkEnv.get.actorSystem. This ActorSystem will contain the configuration
defined in my application jar's reference.conf in both local cluster case,
and in the case I use it directly in an extension to BaseRelation's
local config file that doesn't
exist on your executors?
You might need to email the Spray list.
On Fri, Apr 24, 2015 at 2:38 PM, Yang Lei genia...@gmail.com wrote:
forward to dev.
On Mon, Apr 20, 2015 at 10:46 AM, Yang Lei genia...@gmail.com wrote:
I implemented two kinds
forward to dev.
On Mon, Apr 20, 2015 at 10:46 AM, Yang Lei genia...@gmail.com wrote:
I implemented two kinds of DataSource, one load data during buildScan,
the other returning my RDD class with partition information for future
loading.
My RDD's compute gets actorSystem from
