A couple points: You've got mismatched versions here -- 1.2.0 vs 1.2.1. You should fix that but it's not your problem.
These are also supposed to be 'provided' scope dependencies in Maven. You should get the Scala deps transitively and can import scala.* classes. However, it would be a little bit more correct to depend directly on the scala library classes, but in practice, easiest not to in simple use cases. If you're still having trouble look at the output of "mvn dependency:tree" On Tue, Mar 10, 2015 at 6:32 PM, Mohit Anchlia <mohitanch...@gmail.com> wrote: > I am using maven and my dependency looks like this, but this doesn't seem to > be working > > <dependencies> > > <dependency> > > <groupId>org.apache.spark</groupId> > > <artifactId>spark-streaming_2.10</artifactId> > > <version>1.2.0</version> > > </dependency> > > <dependency> > > <groupId>org.apache.spark</groupId> > > <artifactId>spark-core_2.10</artifactId> > > <version>1.2.1</version> > > </dependency> > > </dependencies> > > > On Tue, Mar 10, 2015 at 11:06 AM, Tathagata Das <t...@databricks.com> wrote: >> >> If you are using tools like SBT/Maven/Gradle/etc, they figure out all the >> recursive dependencies and includes them in the class path. I haven't >> touched Eclipse in years so I am not sure off the top of my head what's >> going on instead. Just in case you only downloaded the >> spark-streaming_2.10.jar then that is indeed insufficient and you have to >> download all the recursive dependencies. May be you should create a Maven >> project inside Eclipse? >> >> TD >> >> On Tue, Mar 10, 2015 at 11:00 AM, Mohit Anchlia <mohitanch...@gmail.com> >> wrote: >>> >>> How do I do that? I haven't used Scala before. >>> >>> Also, linking page doesn't mention that: >>> >>> >>> http://spark.apache.org/docs/1.2.0/streaming-programming-guide.html#linking >>> >>> On Tue, Mar 10, 2015 at 10:57 AM, Sean Owen <so...@cloudera.com> wrote: >>>> >>>> It means you do not have Scala library classes in your project >>>> classpath. >>>> >>>> On Tue, Mar 10, 2015 at 5:54 PM, Mohit Anchlia <mohitanch...@gmail.com> >>>> wrote: >>>> > I am trying out streaming example as documented and I am using spark >>>> > 1.2.1 >>>> > streaming from maven for Java. >>>> > >>>> > When I add this code I get compilation error on and eclipse is not >>>> > able to >>>> > recognize Tuple2. I also don't see any "import scala.Tuple2" class. >>>> > >>>> > >>>> > >>>> > http://spark.apache.org/docs/1.2.0/streaming-programming-guide.html#a-quick-example >>>> > >>>> > >>>> > private void map(JavaReceiverInputDStream<String> lines) { >>>> > >>>> > JavaDStream<String> words = lines.flatMap( >>>> > >>>> > new FlatMapFunction<String, String>() { >>>> > >>>> > @Override public Iterable<String> call(String x) { >>>> > >>>> > return Arrays.asList(x.split(" ")); >>>> > >>>> > } >>>> > >>>> > }); >>>> > >>>> > // Count each word in each batch >>>> > >>>> > JavaPairDStream<String, Integer> pairs = words.map( >>>> > >>>> > new PairFunction<String, String, Integer>() { >>>> > >>>> > @Override public Tuple2<String, Integer> call(String s) throws >>>> > Exception { >>>> > >>>> > return new Tuple2<String, Integer>(s, 1); >>>> > >>>> > } >>>> > >>>> > }); >>>> > >>>> > } >>> >>> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
