Hi Bill, Actually CarbonData is not supported with spark 2.1.1, it is only supported with spark 2.1.0. And about your error it says that method is not present so better give absolute path of CarbonData jar to spark-shell and also check the imports are done properly or not. Please refer to the example CarbonSessionExample inside examples folder for creating carbon session
Regards, Ravindra On Thu, 25 May 2017 at 10:30 AM, Pallavi Singh <[email protected]> wrote: > Hi Bill, > > CarbonContext is only for Spark 1.6, for Spark 2.1 you should use > CarbonSession. Please refer to CarbonSessionExample > <https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonSessionExample.scala> > for > more details. > > On Thu, May 25, 2017 at 12:01 AM, Bill Speirs <[email protected]> wrote: > >> I'm trying to follow the directions for using Spark 2.1.1 with >> CarbonData 1.1.0 found here: >> http://carbondata.apache.org/quick-start-guide.html >> >> I compiled CarbonData using: mvn -DskipTests -Pspark-2.1 >> -Dspark.version=2.1.1 clean package >> >> I ran Spark with: ./bin/spark-shell --jars >> >> ../carbondata-parent-1.1.0/assembly/target/scala-2.11/carbondata_2.11-1.1.0-shade-hadoop2.2.0.jar >> >> I imported the two classes (not sure if the second is a typo with the >> underline): >> >> scala> import org.apache.spark.sql.SparkSession >> import org.apache.spark.sql.SparkSession >> >> scala> import org.apache.spark.sql.CarbonSession._ >> import org.apache.spark.sql.CarbonSession._ >> >> Then tried to create the carbon variable: >> >> scala> val carbon = >> >> SparkSession.builder().config(sc.getConf).getOrCreateCarbonContext("hdfs://localhost:9000/carbondata") >> <console>:28: error: value getOrCreateCarbonContext is not a member of >> org.apache.spark.sql.SparkSession.Builder >> val carbon = >> >> SparkSession.builder().config(sc.getConf).getOrCreateCarbonContext("hdfs://localhost:9000/carbondata") >> >> Thoughts on what I'm doing wrong here? >> >> Bill- >> > > > > -- > Regards | Pallavi Singh > Software Consultant > Knoldus Software LLP > +91-9911235949 >
