Re: Spark job server pros and cons

[Shak S]

Spark job Server(SJS) gives you the ability to have your spark job as a
service.  It has features like caching RDD, publish rest APIs to submit
your job and named RDDs. For more info, refer
https://github.com/spark-jobserver/spark-jobserver. Internally SJS too uses
the same spark job submit so it up to your spark program to deal with
Kafka, Cassandra etc.

Cons --> You need to tweak the settings and configuration if SJS is not
running out of the box. you need to build on your own for production using
sbt, so some Scala knowledge is desirable. It is not completely out of the
box tool, u need to have some learning curve and trouble shooting.


On Fri, Dec 9, 2016 at 4:31 PM, Cassa L  wrote:

> Hi,
> So far, I ran spark jobs directly using spark-submit options.  I have a
> use case to use Spark Job server to run the job. I wanted to find out PROS
> and CONs of using this job server? If anyone can share it, it will be
> great.  My jobs usually connected to multiple data sources like Kafka,
> Custom receiver, Cassandra etc. Will these use cases work as is in job
> server?
>
> Thanks,
> Leena
>

KMediods in Spark java

[Shak S]

Is there any example to implement KMediods cluster in spark and java? I
searched Spark API looks like Spark has not yet implemented KMediods. Any
example or inputs will be appreciated.

Thanks.