[
https://issues.apache.org/jira/browse/IGNITE-7077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16601189#comment-16601189
]
Łukasz Żukowski commented on IGNITE-7077:
-----------------------------------------
Hi
I have question about spark joins in Ignite.
It is possible to do something similiar to joinWithCassandraTable described
below
[https://github.com/datastax/spark-cassandra-connector/blob/master/doc/2_loading.md]
In general I want create query to Ignite Cluster for eache record in
Dataframe/RDD and put result as joined collection.
It is very usefull where we have large dataset but join is only for small
subset of data eg. in streaming.
Or maybe in Ignite Spark integration this can be done in other way.
> Spark Data Frame Support. Strategy to convert complete query to Ignite SQL
> --------------------------------------------------------------------------
>
> Key: IGNITE-7077
> URL: https://issues.apache.org/jira/browse/IGNITE-7077
> Project: Ignite
> Issue Type: New Feature
> Components: spark
> Affects Versions: 2.3
> Reporter: Nikolay Izhikov
> Assignee: Nikolay Izhikov
> Priority: Major
> Labels: bigdata
> Fix For: 2.5
>
>
> Basic support of Spark Data Frame for Ignite implemented in IGNITE-3084.
> We need to implement custom spark strategy that can convert whole Spark SQL
> query to Ignite SQL Query if query consists of only Ignite tables.
> The strategy does nothing if spark query includes not only Ignite tables.
> Memsql implementation can be taken as an example -
> https://github.com/memsql/memsql-spark-connector
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
