Hi,
We have a small mesos cluster and we’d like to be able to initialize some of
our classes, mostly we have a vfs we setup to be allow our code
to access S3, but there doesn’t seem to be any readily obvious way to bootstrap
these kind of classes so that they have the properly initialized
Hi Stephen,
I'm not quite sure what you mean by bootstrapping classes, do you have some
particular examples?
Usually to run any user jar you just need it to be reachable by your slaves
so it can be either S3 or any accessible place, then you just provide your
jar url when you run spark-submit.
Okay my apologies perhaps I’m just not explaining this well.
So we have an application, that creates a spark context and adds all the
necessary jars, the job is submitted to the cluster
and as it runs the code it attempts to pull the data necessary for the job from
s3. We use our own version of
3 matches
Mail list logo
