Re: Spark as a Library

2014-09-16 Thread Daniel Siegmann
ways to connect to Spark? I can’t find in the docs anything other > than using the script. Thanks! > > > > Best, Oliver > > > > *From:* Matei Zaharia [mailto:matei.zaha...@gmail.com] > *Sent:* Tuesday, September 16, 2014 1:31 PM > *To:* Ruebenacker, Oliver A; use

RE: Spark as a Library

2014-09-16 Thread Ruebenacker, Oliver A
script. Thanks! Best, Oliver From: Matei Zaharia [mailto:matei.zaha...@gmail.com] Sent: Tuesday, September 16, 2014 1:31 PM To: Ruebenacker, Oliver A; user@spark.apache.org Subject: Re: Spark as a Library If you want to run the computation on just one machine (using Spark's local mode

Re: Spark as a Library

2014-09-16 Thread Soumya Simanta
It depends on what you want to do with Spark. The following has worked for me. Let the container handle the HTTP request and then talk to Spark using another HTTP/REST interface. You can use the Spark Job Server for this. Embedding Spark inside the container is not a great long term solution IMO b

Re: Spark as a Library

2014-09-16 Thread Matei Zaharia
If you want to run the computation on just one machine (using Spark's local mode), it can probably run in a container. Otherwise you can create a SparkContext there and connect it to a cluster outside. Note that I haven't tried this though, so the security policies of the container might be too