Hi Patrick, Actually, Ignite by itself provides very rich SQL support [1]. So it sounds like you can achieve your goals even without Spark: save all the data in Ignite cluster, probably with persistence enabled [2] and run SQL against this data. You will be able to create any arbitrary indexes and run very fast ad-hoc queries.
If you still want to use Spark DataFrame/Dataset APIs to do the analytics, then you would really benefit from the upcoming integration between Ignite and Spark DataFrame framework [3]. To my knowledge, it's currently in development, so hopefully will be available in one of the nearest releases. Let me know if you have more questions! [1] https://apacheignite.readme.io/docs/distributed-sql [2] https://apacheignite.readme.io/docs/distributed-persistent-store [3] https://issues.apache.org/jira/browse/IGNITE-3084 -Val -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
