I'm trying to follow the directions for using Spark 2.1.1 with
CarbonData 1.1.0 found here:
http://carbondata.apache.org/quick-start-guide.html
I compiled CarbonData using: mvn -DskipTests -Pspark-2.1
-Dspark.version=2.1.1 clean package
I ran Spark with: ./bin/spark-shell --jars
../carbondata-parent-1.1.0/assembly/target/scala-2.11/carbondata_2.11-1.1.0-shade-hadoop2.2.0.jar
I imported the two classes (not sure if the second is a typo with the
underline):
scala> import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.SparkSession
scala> import org.apache.spark.sql.CarbonSession._
import org.apache.spark.sql.CarbonSession._
Then tried to create the carbon variable:
scala> val carbon =
SparkSession.builder().config(sc.getConf).getOrCreateCarbonContext("hdfs://localhost:9000/carbondata")
<console>:28: error: value getOrCreateCarbonContext is not a member of
org.apache.spark.sql.SparkSession.Builder
val carbon =
SparkSession.builder().config(sc.getConf).getOrCreateCarbonContext("hdfs://localhost:9000/carbondata")
Thoughts on what I'm doing wrong here?
Bill-