Yes, It was a dependency issue. I was using incompatible versions. The command *mvn dependency:tree -Dverbose *helped me fix this.
Cheers Asfandyar Ashraf Malik Mobile: +49 15751174449 <%2B49%20151%20230%20130%2066> Email: asfand...@kreditech.com <%2B49%20151%20230%20130%2066> Kreditech Holding SSL GmbH Ludwig-Erhard-Straße 1, 20459 Hamburg, Germany 2016-06-12 18:36 GMT+02:00 Ted Yu <yuzhih...@gmail.com>: > What's the value of spark.version ? > > Do you know which version of Spark mongodb connector 0.10.3 was built > against ? > > You can use the following command to find out: > mvn dependency:tree > > Maybe the Spark version you use is different from what mongodb connector > was built against. > > On Fri, Jun 10, 2016 at 2:50 AM, Asfandyar Ashraf Malik < > asfand...@kreditech.com> wrote: > >> Hi, >> I did not notice that I put it twice. >> I changed that and ran my program but it still gives the same error: >> >> java.lang.NoSuchMethodError: >> org.apache.spark.sql.catalyst.ScalaReflection$.typeOfObject()Lscala/PartialFunction; >> >> >> Cheers >> >> >> >> 2016-06-10 11:47 GMT+02:00 Alonso Isidoro Roman <alons...@gmail.com>: >> >>> why *spark-mongodb_2.11 dependency is written twice in pom.xml?* >>> >>> Alonso Isidoro Roman >>> [image: https://]about.me/alonso.isidoro.roman >>> >>> <https://about.me/alonso.isidoro.roman?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links> >>> >>> 2016-06-10 11:39 GMT+02:00 Asfandyar Ashraf Malik < >>> asfand...@kreditech.com>: >>> >>>> Hi, >>>> I am using Stratio library to get MongoDB to work with Spark but I get >>>> the following error: >>>> >>>> java.lang.NoSuchMethodError: >>>> org.apache.spark.sql.catalyst.ScalaReflection >>>> >>>> This is my code. >>>> >>>> --------------------------------------------------------------------------------------- >>>> * public static void main(String[] args) {* >>>> >>>> * JavaSparkContext sc = new JavaSparkContext("local[*]", "test >>>> spark-mongodb java"); * >>>> * SQLContext sqlContext = new SQLContext(sc); * >>>> >>>> * Map options = new HashMap(); * >>>> * options.put("host", "xyz.mongolab.com:59107 >>>> <http://xyz.mongolab.com:59107>"); * >>>> * options.put("database", "heroku_app3525385");* >>>> * options.put("collection", "datalog");* >>>> * options.put("credentials", "*****,****,****");* >>>> >>>> * DataFrame df = >>>> sqlContext.read().format("com.stratio.datasource.mongodb").options(options).load();* >>>> * df.registerTempTable("datalog"); * >>>> * df.show();* >>>> >>>> * }* >>>> >>>> --------------------------------------------------------------------------------------- >>>> My pom file is as follows: >>>> >>>> *<dependencies>* >>>> * <dependency>* >>>> * <groupId>org.apache.spark</groupId>* >>>> * <artifactId>spark-core_2.11</artifactId>* >>>> * <version>${spark.version}</version>* >>>> * </dependency>* >>>> * <dependency>* >>>> * <groupId>org.apache.spark</groupId>* >>>> * <artifactId>spark-catalyst_2.11 </artifactId>* >>>> * <version>${spark.version}</version>* >>>> * </dependency>* >>>> * <dependency>* >>>> * <groupId>org.apache.spark</groupId>* >>>> * <artifactId>spark-sql_2.11</artifactId>* >>>> * <version>${spark.version}</version>* >>>> * </dependency> * >>>> * <dependency>* >>>> * <groupId>com.stratio.datasource</groupId>* >>>> * <artifactId>spark-mongodb_2.11</artifactId>* >>>> * <version>0.10.3</version>* >>>> * </dependency>* >>>> * <dependency>* >>>> * <groupId>com.stratio.datasource</groupId>* >>>> * <artifactId>spark-mongodb_2.11</artifactId>* >>>> * <version>0.10.3</version>* >>>> * <type>jar</type>* >>>> * </dependency>* >>>> * </dependencies>* >>>> >>>> >>>> Regards >>>> >>> >>> >> >
