Re: Sharing Spark Drivers

2015-02-24 Thread Chip Senkbeil
arted-with-the-Spark-Kernel Signed, Chip Senkbeil On Tue Feb 24 2015 at 8:04:08 AM John Omernik wrote: > I have been posting on the Mesos list, as I am looking to see if it > it's possible or not to share spark drivers. Obviously, in stand > alone cluster mode, the Master handles re

Re: spark driver behind firewall

2015-02-06 Thread Chip Senkbeil
send Scala code to interact with Apache Spark. You would still need to expose the Spark Kernel outside the firewall (similar to Kostas' suggestion about the jobserver), of course. Signed, Chip Senkbeil On Thu Feb 05 2015 at 11:07:28 PM Kostas Sakellis wrote: > Yes, the driver has to be

Re: How to design a long live spark application

2015-02-05 Thread Chip Senkbeil
: https://github.com/ibm-et/spark-kernel/wiki/Getting-Started-with-the-Spark-Kernel If you have any more questions about the project, feel free to email me! Signed, Chip Senkbeil On Thu Feb 05 2015 at 10:58:01 AM Corey Nolet wrote: > Here's another lightweight example of running a SparkCont

Re: SVD in pyspark ?

2015-01-25 Thread Chip Senkbeil
message protocol - used by IPython 3.0 - to communicate. See the getting started section here: https://github.com/ibm-et/spark-kernel/wiki/Getting-Started-with-the-Spark-Kernel It discusses getting IPython connected to a Spark Kernel. If you have any more questions, feel free to ask! Signed, Chip