Thanks Costin.
Il giorno lunedì 20 aprile 2015 12:38:30 UTC+2, Costin Leau ha scritto: > > Beta3 work with Spark SQL 1.0 and 1.1 > Spark SQL 1.2 was released after that and broke binary backwards > compatibility however his has been fixed in master/dev > version [1] > Note that Spark SQL 1.3 was released as well and again, broke backwards > compatibility this time significant hence why > there are now two versions [2] - make sure to use the one appropriate for > your Spark version. > > Cheers, > > [1] > http://www.elastic.co/guide/en/elasticsearch/hadoop/master/install.html#download-dev > > [2] > http://www.elastic.co/guide/en/elasticsearch/hadoop/master/spark.html#spark-sql-versions > > > On 4/20/15 1:17 PM, michele crudele wrote: > > > > I wrote this simple notebook in scala using Elasticsearch Spark adapter: > > > > %AddJar > file:///tools/elasticsearch-hadoop-2.1.0.Beta3/dist/elasticsearch-spark_2.10-2.1.0.BUILD-SNAPSHOT.jar > > > > %AddJar > file:///tools/elasticsearch-hadoop-2.1.0.Beta3/dist/elasticsearch-hadoop-2.1.0.BUILD-SNAPSHOT.jar > > > > > > > > // Write something to spark/docs index > > // > > import org.elasticsearch.spark._ > > > > val michelangelo = Map("artist" -> "Michelangelo", "bio" -> "Painter, > sculptor, architect born in Florence in... ") > > val leonardo = Map("artist" -> "Leonardo", "bio" -> "Absolute genius; > painter, inventor born in the little village of > > Vinci...") > > > > sc.makeRDD(Seq(michelangelo, leonardo)).saveToEs("spark/docs") > > > > // Search for painters through spark/docs > > // > > val painters = sc.esRDD("spark/docs", "?q=painter") > > println("Number of painters in spark/docs: " + painters.count()) > > > > painters.collect().foreach(x => println("ID: " + x._1)) > > > > // Try to read using SparkSQL > > // > > import org.apache.spark.sql.SQLContext > > import org.elasticsearch.spark._ > > import org.elasticsearch.spark.sql._ > > > > val sql = new SQLContext(sc) > > > > // Here is where I get an exception > > // > > val docsSql = sql.esRDD("spark/docs") > > > > > > Name: java.lang.NoSuchMethodError > > Message: > > > org.apache.spark.sql.catalyst.types.StructField.<init>(Ljava/lang/String;Lorg/apache/spark/sql/catalyst/types/DataType;Z)V > > > > StackTrace: > > > org.elasticsearch.spark.sql.MappingUtils$.org$elasticsearch$spark$sql$MappingUtils$$convertField(MappingUtils.scala:75) > > > > > org.elasticsearch.spark.sql.MappingUtils$$anonfun$convertToStruct$1.apply(MappingUtils.scala:54) > > > > > org.elasticsearch.spark.sql.MappingUtils$$anonfun$convertToStruct$1.apply(MappingUtils.scala:54) > > > > > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) > > > > > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) > > > > > scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) > > > > scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108) > > scala.collection.TraversableLike$class.map(TraversableLike.scala:244) > > scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:108) > > > org.elasticsearch.spark.sql.MappingUtils$.convertToStruct(MappingUtils.scala:54) > > > > > org.elasticsearch.spark.sql.MappingUtils$.discoverMapping(MappingUtils.scala:47) > > > > org.elasticsearch.spark.sql.EsSparkSQL$.esRDD(EsSparkSQL.scala:27) > > org.elasticsearch.spark.sql.EsSparkSQL$.esRDD(EsSparkSQL.scala:23) > > > org.elasticsearch.spark.sql.package$SQLContextFunctions.esRDD(package.scala:16) > > > > > $line460.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$$$$$c57ec8bf9b0d5f6161b97741d596ff0$$$$wC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:111) > > > > > $line460.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:116) > > > > > $line460.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:118) > > > > > $line460.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$ > > > > > > > > > > > > What am I doing wrong? Thanks > > - Michele > > > > -- > > You received this message because you are subscribed to the Google > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to > > elasticsearc...@googlegroups.com <javascript:> <mailto: > elasticsearch+unsubscr...@googlegroups.com <javascript:>>. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/elasticsearch/49a5c624-bda0-4835-98bc-915ec80d4fa3%40googlegroups.com > > > < > https://groups.google.com/d/msgid/elasticsearch/49a5c624-bda0-4835-98bc-915ec80d4fa3%40googlegroups.com?utm_medium=email&utm_source=footer>. > > > > For more options, visit https://groups.google.com/d/optout. > > -- > Costin > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fbbc4fc9-6662-4d57-b189-cbf8c60f36af%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
