Hi,
I'm just trying out Apache Ignite and have issues when trying to access the
cache by using SQL in Spark.
I have a simple data container defined:
case class DataContainer(@QuerySqlField key: Int, @QuerySqlField value:
String) extends Serializable{
override def toString = s"Key$key, value $value"
}
Then I put some data into the cache via a Spark Batch app:
val dataContainerArray = Array[DataContainer](new DataContainer(1,
"T"), new DataContainer(2, "A"))
val dataContainerRdd = sc.parallelize(dataContainerArray)
val dataKeyValueRdd = dataContainerRdd.map(item => Pair(item.key, item))
//configure the cache...
val ccfg = new CacheConfiguration[Int, DataContainer]()
ccfg.setIndexedTypes(Int.getClass, DataContainer.getClass)
ccfg.setName("C2")
val cacheRdd = igniteContext.fromCache(ccfg)
cacheRdd.savePairs(dataKeyValueRdd, true)
Now, I am trying to read the cache from another Spark app
val igniteContext = new IgniteContext[Int, DataContainer](sc,
() => new IgniteConfiguration())
//get the cache as a rdd representation
val cacheRdd = igniteContext.fromCache("C2")
// get the entry for key = 1 (WORKS FINE)
val result = cacheRdd.filter(_._1 == 1).collect()
result.foreach(row => println(row._2))
//no result :-(
cacheRdd.sql("select * from
C2.DATACONTAINER_").foreach(row=>println(row.toString()))
The access by the filter function(RDD means) works fine. Unfortunately, the
SQL access does not return any results. Does anybody have an idea what I am
making wrong here ?
Thanks,
Marco