Re: Error: at sqlContext.createDataFrame with RDD and Schema

2016-12-28 Thread Liang-Chi Hsieh
Your schema is all fields are string: > val stdSchema= StructType(stdSchemaString.split(",").map(fieldName => > StructField(fieldName, StringType, true))) But looks like you have integer columns in the RDD? Chetan Khatri wrote > Resolved above error by creating SparkSession > > val spark = Sp

Re: Error: at sqlContext.createDataFrame with RDD and Schema

2016-12-28 Thread Chetan Khatri
Resolved above error by creating SparkSession val spark = SparkSession.builder().appName("Hbase - Spark POC").getOrCreate() Error after: spark.sql("SELECT * FROM student").show() But while doing show() action on Dataframe throws below error: scala> sqlContext.sql("select * from student").show(

Error: at sqlContext.createDataFrame with RDD and Schema

2016-12-28 Thread Chetan Khatri
Hello Spark Community, I am reading HBase table from Spark and getting RDD but now i wants to convert RDD of Spark Rows and want to convert to DF. *Source Code:* bin/spark-shell --packages it.nerdammer.bigdata:spark-hbase-connector_2.10:1.0.3 --conf spark.hbase.host=127.0.0.1 import it.nerdamme