I am using maven and my dependency looks like this, but this doesn't seem to be working
<dependencies> <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-streaming_2.10</artifactId> <version>1.2.0</version> </dependency> <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-core_2.10</artifactId> <version>1.2.1</version> </dependency> </dependencies> On Tue, Mar 10, 2015 at 11:06 AM, Tathagata Das <t...@databricks.com> wrote: > If you are using tools like SBT/Maven/Gradle/etc, they figure out all the > recursive dependencies and includes them in the class path. I haven't > touched Eclipse in years so I am not sure off the top of my head what's > going on instead. Just in case you only downloaded the > spark-streaming_2.10.jar then that is indeed insufficient and you have to > download all the recursive dependencies. May be you should create a Maven > project inside Eclipse? > > TD > > On Tue, Mar 10, 2015 at 11:00 AM, Mohit Anchlia <mohitanch...@gmail.com> > wrote: > >> How do I do that? I haven't used Scala before. >> >> Also, linking page doesn't mention that: >> >> >> http://spark.apache.org/docs/1.2.0/streaming-programming-guide.html#linking >> >> On Tue, Mar 10, 2015 at 10:57 AM, Sean Owen <so...@cloudera.com> wrote: >> >>> It means you do not have Scala library classes in your project classpath. >>> >>> On Tue, Mar 10, 2015 at 5:54 PM, Mohit Anchlia <mohitanch...@gmail.com> >>> wrote: >>> > I am trying out streaming example as documented and I am using spark >>> 1.2.1 >>> > streaming from maven for Java. >>> > >>> > When I add this code I get compilation error on and eclipse is not >>> able to >>> > recognize Tuple2. I also don't see any "import scala.Tuple2" class. >>> > >>> > >>> > >>> http://spark.apache.org/docs/1.2.0/streaming-programming-guide.html#a-quick-example >>> > >>> > >>> > private void map(JavaReceiverInputDStream<String> lines) { >>> > >>> > JavaDStream<String> words = lines.flatMap( >>> > >>> > new FlatMapFunction<String, String>() { >>> > >>> > @Override public Iterable<String> call(String x) { >>> > >>> > return Arrays.asList(x.split(" ")); >>> > >>> > } >>> > >>> > }); >>> > >>> > // Count each word in each batch >>> > >>> > JavaPairDStream<String, Integer> pairs = words.map( >>> > >>> > new PairFunction<String, String, Integer>() { >>> > >>> > @Override public Tuple2<String, Integer> call(String s) throws >>> Exception { >>> > >>> > return new Tuple2<String, Integer>(s, 1); >>> > >>> > } >>> > >>> > }); >>> > >>> > } >>> >> >> >
