You could try the following: create a minimal project using sbt or Maven, add spark-streaming-twitter as a dependency, run sbt assembly (or mvn package) on that to create a fat jar (with Spark as provided dependency), and add that to the shell classpath when starting up.
On Tue, Jul 15, 2014 at 9:06 AM, Praveen Seluka <psel...@qubole.com> wrote: > If you want to make Twitter* classes available in your shell, I believe > you could do the following > 1. Change the parent pom module ordering - Move external/twitter before > assembly > 2. In assembly/pom.xm, add external/twitter dependency - this will package > twitter* into the assembly jar > > Now when spark-shell is launched, assembly jar is in classpath - hence > twitter* too. I think this will work (remember trying this sometime back) > > > On Tue, Jul 15, 2014 at 11:59 AM, Nicholas Chammas < > nicholas.cham...@gmail.com> wrote: > >> Hmm, I'd like to clarify something from your comments, Tathagata. >> >> Going forward, is Twitter Streaming functionality not supported from the >> shell? What should users do if they'd like to process live Tweets from the >> shell? >> >> Nick >> >> >> On Mon, Jul 14, 2014 at 11:50 PM, Nicholas Chammas < >> nicholas.cham...@gmail.com> wrote: >> >>> At some point, you were able to access TwitterUtils from spark shell >>>> using Spark 1.0.0+ ? >>> >>> >>> Yep. >>> >>> >>>> If yes, then what change in Spark caused it to not work any more? >>> >>> >>> It still works for me. I was just commenting on your remark that it >>> doesn't work through the shell, which I now understand to apply to versions >>> of Spark before 1.0.0. >>> >>> Nick >>> >> >> >
