See if you can import scala libraries in your project. On Tue, Mar 10, 2015 at 11:32 AM, Mohit Anchlia <mohitanch...@gmail.com> wrote:
> I am using maven and my dependency looks like this, but this doesn't seem > to be working > > <dependencies> > > <dependency> > > <groupId>org.apache.spark</groupId> > > <artifactId>spark-streaming_2.10</artifactId> > > <version>1.2.0</version> > > </dependency> > > <dependency> > > <groupId>org.apache.spark</groupId> > > <artifactId>spark-core_2.10</artifactId> > > <version>1.2.1</version> > > </dependency> > > </dependencies> > > On Tue, Mar 10, 2015 at 11:06 AM, Tathagata Das <t...@databricks.com> > wrote: > >> If you are using tools like SBT/Maven/Gradle/etc, they figure out all the >> recursive dependencies and includes them in the class path. I haven't >> touched Eclipse in years so I am not sure off the top of my head what's >> going on instead. Just in case you only downloaded the >> spark-streaming_2.10.jar then that is indeed insufficient and you have to >> download all the recursive dependencies. May be you should create a Maven >> project inside Eclipse? >> >> TD >> >> On Tue, Mar 10, 2015 at 11:00 AM, Mohit Anchlia <mohitanch...@gmail.com> >> wrote: >> >>> How do I do that? I haven't used Scala before. >>> >>> Also, linking page doesn't mention that: >>> >>> >>> http://spark.apache.org/docs/1.2.0/streaming-programming-guide.html#linking >>> >>> On Tue, Mar 10, 2015 at 10:57 AM, Sean Owen <so...@cloudera.com> wrote: >>> >>>> It means you do not have Scala library classes in your project >>>> classpath. >>>> >>>> On Tue, Mar 10, 2015 at 5:54 PM, Mohit Anchlia <mohitanch...@gmail.com> >>>> wrote: >>>> > I am trying out streaming example as documented and I am using spark >>>> 1.2.1 >>>> > streaming from maven for Java. >>>> > >>>> > When I add this code I get compilation error on and eclipse is not >>>> able to >>>> > recognize Tuple2. I also don't see any "import scala.Tuple2" class. >>>> > >>>> > >>>> > >>>> http://spark.apache.org/docs/1.2.0/streaming-programming-guide.html#a-quick-example >>>> > >>>> > >>>> > private void map(JavaReceiverInputDStream<String> lines) { >>>> > >>>> > JavaDStream<String> words = lines.flatMap( >>>> > >>>> > new FlatMapFunction<String, String>() { >>>> > >>>> > @Override public Iterable<String> call(String x) { >>>> > >>>> > return Arrays.asList(x.split(" ")); >>>> > >>>> > } >>>> > >>>> > }); >>>> > >>>> > // Count each word in each batch >>>> > >>>> > JavaPairDStream<String, Integer> pairs = words.map( >>>> > >>>> > new PairFunction<String, String, Integer>() { >>>> > >>>> > @Override public Tuple2<String, Integer> call(String s) throws >>>> Exception { >>>> > >>>> > return new Tuple2<String, Integer>(s, 1); >>>> > >>>> > } >>>> > >>>> > }); >>>> > >>>> > } >>>> >>> >>> >> >
