now I get a run time error...
error: Unable to find encoder for type stored in a Dataset. Primitive
types (Int, String, etc) and Product types (case classes) are supported by
importing spark.implicits._ Support for serializing other types will be
added in future releases.
[ERROR] val userDS:Dataset[Teamuser] = userDF.as[Teamuser]
On Thu, Mar 23, 2017 at 12:49 PM, shyla deshpande <deshpandesh...@gmail.com>
wrote:
> I realized, my case class was inside the object. It should be defined
> outside the scope of the object. Thanks
>
> On Wed, Mar 22, 2017 at 6:07 PM, shyla deshpande <deshpandesh...@gmail.com
> > wrote:
>
>> Why userDS is Dataset[Any], instead of Dataset[Teamuser]? Appreciate your
>> help. Thanks
>>
>> val spark = SparkSession
>> .builder
>> .config("spark.cassandra.connection.host", cassandrahost)
>> .appName(getClass.getSimpleName)
>> .getOrCreate()
>>
>> import spark.implicits._
>> val sqlContext = spark.sqlContext
>> import sqlContext.implicits._
>>
>> case class Teamuser(teamid:String, userid:String, role:String)
>> spark
>> .read
>> .format("org.apache.spark.sql.cassandra")
>> .options(Map("keyspace" -> "test", "table" -> "teamuser"))
>> .load
>> .createOrReplaceTempView("teamuser")
>>
>> val userDF = spark.sql("SELECT teamid, userid, role FROM teamuser")
>>
>> userDF.show()
>>
>> val userDS:Dataset[Teamuser] = userDF.as[Teamuser]
>>
>>
>
