I'm trying to get a few columns from a Cassandra table from Spark and put
them in a case class. If I want all the columns that I have in my case
class works. But, I only want to bring a few of them and don't have a
specific case class for each case.
I tried to overload constructor in the case class and define a normal class
but I didn't get to work.
//It doesn't work, it's normal because there aren't an specific contructor.
case class Father(idPadre: Int, name: String, lastName: String,
children: Map[Int,Son], hobbies: Map[Int,Hobbie], lastUpdate: Date)
//It works, because it has the right contructor. I tried to do an
companion object and def others contructors but it didn't work
case class FatherEspecifica(idFather: Int, name: String, children: Map[Int,Son])
//Problems in compilation, I don't know why.
class FatherClaseNormal(idFather: Int, name: String, lastName: String,
children: Map[Int,Son], hobbies: Map[Int,Hobbie], lastUpdate: Date){
/**
* A secondary constructor.
*/
def this(name: String) {
this(0, name, "", Map(), Map(), new Date());
println("\nNo last name or age given.")
}
}
//I'm trying to get some a few columns and don't have to have all the
case classes and I would like to map directly to case class and don't
use CassandraRows.
joinRdd = rddAvro.joinWithCassandraTable[FatherXXX]("poc_udt",
"father",SomeColumns("id_father", "name", "children"))
CREATE TABLE IF NOT EXISTS poc_udt.father(
id_father int PRIMARY KEY,
name text,
last_name text,
children map<int,frozen<son>>,
hobbies map<int,frozen<hobbie>>,
last_update timestamp
)
When I use a normal class the error is:
Error:(57, 67) No RowReaderFactory can be found for this type
Error occurred in an application involving default arguments.
val joinRdd =
rddAvro.joinWithCassandraTable[FatherClaseNormal]("poc_udt",
"father",SomeColumns("name"))
