If you want to make Twitter* classes available in your shell, I believe you could do the following 1. Change the parent pom module ordering - Move external/twitter before assembly 2. In assembly/pom.xm, add external/twitter dependency - this will package twitter* into the assembly jar
Now when spark-shell is launched, assembly jar is in classpath - hence twitter* too. I think this will work (remember trying this sometime back) On Tue, Jul 15, 2014 at 11:59 AM, Nicholas Chammas < nicholas.cham...@gmail.com> wrote: > Hmm, I'd like to clarify something from your comments, Tathagata. > > Going forward, is Twitter Streaming functionality not supported from the > shell? What should users do if they'd like to process live Tweets from the > shell? > > Nick > > > On Mon, Jul 14, 2014 at 11:50 PM, Nicholas Chammas < > nicholas.cham...@gmail.com> wrote: > >> At some point, you were able to access TwitterUtils from spark shell >>> using Spark 1.0.0+ ? >> >> >> Yep. >> >> >>> If yes, then what change in Spark caused it to not work any more? >> >> >> It still works for me. I was just commenting on your remark that it >> doesn't work through the shell, which I now understand to apply to versions >> of Spark before 1.0.0. >> >> Nick >> > >
