Hi,
I'm working on Cassandra-Spark integration and I hit a pretty severe
problem. One of the provided functionality is mapping Cassandra rows into
objects of user-defined classes. E.g. like this:
class MyRow(val key: String, val data: Int)
sc.cassandraTable("keyspace", "table").select("key", "data").as[MyRow] //
returns CassandraRDD[MyRow]
In this example CassandraRDD creates MyRow instances by reflection, i.e.
matches selected fields from Cassandra table and passes them to the
constructor.
Unfortunately this does not work in Spark REPL.
Turns out any class declared on the REPL is an inner classes, and to be
successfully created, it needs a reference to the outer object, even though
it doesn't really use anything from the outer context.
scala> class SomeClass
defined class SomeClass
scala> classOf[SomeClass].getConstructors()(0)
res11: java.lang.reflect.Constructor[_] = public
$iwC$$iwC$SomeClass($iwC$$iwC)
I tried passing a null as a temporary workaround, and it also doesn't work
- I get NPE.
How can I get a reference to the current outer object representing the
context of the current line?
Also, plain non-spark Scala REPL doesn't exhibit this behaviour - and
classes declared on the REPL are proper top-most classes, not inner ones.
Why?
Thanks,
Piotr
--
Piotr Kolaczkowski, Lead Software Engineer
pkola...@datastax.com
777 Mariners Island Blvd., Suite 510
San Mateo, CA 94404
