Nick,
Thanks - I was already nudged in this direction ;) but was hoping to see
if I can roll my own.
Ognen
On 2/23/14, 2:43 PM, Nick Pentreath wrote:
You might want to try out the Spark job server PR:
https://github.com/apache/incubator-spark/pull/222
It could be used for your use case. You can write the relevant code,
upload the jars, and run it via passing along the conf.
You could then use a very lightweight scalatra servlet on the frontend
that executes request to this backend job server.
—
Sent from Mailbox <https://www.dropbox.com/mailbox> for iPhone
On Sun, Feb 23, 2014 at 10:15 PM, Ognen Duzlevski
<og...@nengoiksvelzud.com <mailto:og...@nengoiksvelzud.com>> wrote:
On 2/23/14, 10:26 AM, Ognen Duzlevski wrote:
Hello all,
perhaps too ambitiously ;) I have decided to try and roll my own
Scalatra app that connects to a spark cluster and executes a
query when a certain URL is accessed - I am just trying to figure
out how to get these things going.
Is there anything I should pay particular attention to? I am
running the scalatra servlet on the same machine spark standalone
is running, only the scalatra servlet is listening to port 8082.
I have made some headway in this, I think. Unfortunately, I am too
new to scala and sbt and the whole ecosystem so what I may say
next may sound like rubbish ;). The problem seems to be that
scalatra is at version 2.2.2 which comes with Akka 2.1.3 and
Spark-0.9 is at akka 2.2.3. Apparently things have changed in Akka
between the two versions. I am still unable to get this going.
Even if I downgrade spark to 0.8.1 - it comes with Akka 2.0.5 and
scalatra will be ahead.
Does anyone have an idea how to put the two together?
Thanks!
Ognen
--
Some people, when confronted with a problem, think "I know, I'll use regular
expressions." Now they have two problems.
-- Jamie Zawinski
